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ABSTRACT 


Physical  scientists  at  two  Air  Force  research  centers  were  intensively  interviewed 
concerning  the  nature  of  scientific  productivity  and  the  characteristics  of  effective  scien¬ 
tists.  Based  on  these  interview  suggestions,  data  were  collected  on  52  criteria.  These 
were  reduced  analytically  to  14  factor  scores.  Following  this,  several  tests  and  question¬ 
naires  were  developed  for  tryout  as  predictors.  Scores  from  these  and  previously  developed 
instruments  that  showed  promise  were  correlated  with  the  factor  scores  and  three  of  the 
original  criteria.  In  the  sample  of  107  scientists,  criteria  most  related  to  tests  and  question¬ 
naires  (in  terms  of  number  of  significant  correlations)  were  ratings  of  likableness  as  a  member 
of  a  research  team,  membership  in  professional  societies,  organizational  status,  rated  work 
output,  supervisory  ratings  on  overall  performance,  and  peer  rankings  on  overall  productivity. 
The  instruments  that  had  scores  correlating  with  the  greatest  number  of  criteria  were  a  bio¬ 
graphical  data  questionnaire,  self-ratings,  and  a  questionnaire  desiqned  to  measure  minimum 
level  of  aspiration.  The  outcomes  of  this  investigation  were  identification  of  a  wide  variety 
of  measurable  criteria  and  a  number  of  self-report  instruments  suitable  for  future  longitudinal 
followup  and  validation  as  a  means  of  identifying  kinds  of  scientific  tale  t  needed  by  the 
Air  Force. 
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EXPLORATIONS  IN  THE  MEASUREMENT  AND  PREDICTION 
OF  CONTRIBUTIONS  OF  ONE  SAMPLE  OF  SCIENTISTS1 

The  current  demond  for  scientific  proqress  in  the  notion  hos  stimuloted  an  increosing  inter¬ 
est  in  the  nature  of  scientific  tolent  ond  its  identification,  development,  ond  utilization.  During 
the  past  few  years,  severol  researchers  (Cottell,  1959;  Mullins,  1 000;  Roe,  1958;  Sounders,  1956; 
Stein,  1956;  D  W.  Taylor,  1958)  have  completed  one  or  more  explorotory  studies  on  some  ospect  of 
scientific  tolent,  especiolly  pertoining  to  creotivity  and  productivity  in  scientists.  This  report 
covers  o  three-yeor  project  designed  to  determine  methods  of  predicting  productivity  and  creotivity 
in  Air  Force  bosic  research  laboratories.  The  work  progressed  in  six  phoses:  (o)  exploration  of 
the  problem  of  meosunng  the  extent  to  which  scientists  ore  making  contributions  to  the  various 
phoses  of  the  work  in  the  Air  Force  reseorch  ond  development  progrom  to  which  they  are  expected  to 
contribute;  (b)  development  of  methods  of  meosuring  or  ossessing  the  voriobles  defined  in  the 
previous  phose  ond  collection  of  such  criterion  dato;  (c)  explorotion  of  methods  for  identifying 
importont  personol  chorocteristics  which  differentiate  between  productive  ond  nonproductive 
scientists;  (d)  development  of  tests  and  other  devices  to  measure  the  voriables  hypothesized  in 
the  previous  phose;  (e)  administration  ond  anolysis  of  the  tests;  and  (f)  selection  of  o  bottery  of 
instruments  which  predict  scientific  productivity  ond  provide  o  psychologicol  model  of  the  produc¬ 
tive  scientist. 

Two  major  studies  were  to  be  accomplished  in  this  project.  The  first  one  wos  to  deal  with 
the  problem  of  creotive,  productive,  ond  other  contributions  of  scientists.  The  second  study  v/os 
to  develop  meosures  of  psychologicol  chorocteristics  presumed  to  be  importont  in  science  ond  to 
validote  these  meosures  oqoinst  the  criterio  developed  in  the  first  study  of  the  project.  A  large 
interviewing  octivity  preceded  these  two  moin  studies.  The  interviews  were  designed  to  gother 
suggestions  from  o  lorge  somple  of  scientists  on  the  various  types  of  contributions  of  scientists 
ond  on  the  psycholoqicol  chorocteristics  thot  moy  be  volid  as  predictors  of  one  or  more  types  of 
these  contributions  of  scientists. 

Relevont  studies  in  the  oreo,  such  os  those  by  Chorness  (1955),  Pelz  (1955),  ond  Stein 
(1957),  were  considered  when  constructing  instruments  for  collecting  the  criterion  doto  on  scien¬ 
tists.  These  suggested  criterio  included  possible  woys  for  estimoting  o  person's  copocity  for 
(a)  fulfilling  functions  as  a  scientist,  (b)  fulfilling  functions  as  on  employee  of  on  orgonizotion, 

(c)  fulfilling  functions  os  a  member  of  o  profession,  (d)  exercising  leodership  in  reseorch,  ond 
(e)  promoting  the  ideos  of  others. 

These  studies  and  most  other  recent  reseorch  studies  on  creotivity  ond  productivity  of  scien¬ 
tists  hove  been  reported  ond  considered  in  the  University  of  Utoh  series  of  reseorch  conferences 
on  "The  Identification  of  Creotive  Scientific  Tolent"  (C. W.  Taylor,  1956,  1958,  ond  1959).  These 
conference  reports  hove  provided  one  of  the  moni  boses  from  which  this  lorg^-scole  reseorch  pro¬ 
ject  on  scientists  in  reseorch  loborotories  hos  been  undertoken. 

Criteria  of  success  in  science  hove  not  been  well  understood  because  few  comprehensive 
scientific  investigations  of  criterio  hove  been  undertoken.  Uncriticol  judgments  might  lead 
investigotors  into  o  consideration  of  publications,  potents,  or  other  obvious  tongible  products  as 
the  sole  criterio  of  success,  but  ofter  closely  onolyzing  scientific  productivity  it  oppeored  thot 
other  contributions  ore  required  of  scientists  thot  would  olso  be  colled  productive.  The  word 
"contributions"  wos  chosen  in  this  reseorch  project  when  referring  to  productive  ond  creotive 
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acts  since  it  is  broader  in  connotation,  including  the  fruitfulness  of  scientists  in  various  roles 
and  activities,  and  is  better  understood  during  communications  wi  h  the  scientists  than  either 
productivity  or  creativity. 

From  an  intensive  review  of  the  related  work  the  investigate  s  found  little  information 
conclusive  enough  to  justify  adopting  dogmatically  any  single  criterion  or  group  of  criteria 
directly.  The  studies  reviewed  represented  attempts  to  better  understand  the  nature  of  the 
criteria  of  productivity  and  creativity  in  science,  but  at  the  same  time  were  not  definitive  or 
comprehensive  enough  to  satisfy  all  of  the  needs  of  the  present  study.  An  attempt  was  made  in 
this  research  to  identify  and  measure  all  of  the  possible  criteria  of  success  in  science  that  might 
lend  ihemselves  to  analysis  in  the  experimental  framework  of  the  project. 

The  initial  approach  involved  the  conducting  and  analyzing  of  over  200  extensive  interviews 
with  scientists  in  two  research  centers.  2  Scientists'  statements  about  criteria  of  contributions 
were  analyzed  and  organized  to  yield  a  list  of  about  150  criterion  scores  which  could  be  obtained 
from  criterion  instruments.  Means  for  collecting  data  on  these  contributions  in  science  were 
devised  and  complete  criterion  and  predictor  data  were  collected  on  the  large  majority  of  the 
scientists  in  one  directorate  at  one  basic  research  Air  Force  center. 

This  report  will  first  present  the  intensive  interviewing  study  and  analysis,  followed  by  the 
study  of  the  multiple  contributions  of  scientists.  The  predict  study,  which  was  designed  to 
identify  psychological  characteristics  related  to  each  type  or  contribution  of  scientists,  will  be 
reported  last  in  the  seguence. 

THE  INTERVIEW  STUDY  AND  INTERVIEW  ANALYSES 

The  interview  study  admittedly  entailed  very  subjective  approaches,  which  nonetheless 
helped  considerably  in  the  design  of  the  methodology  for  criterion  data  collection.  Three  mem¬ 
bers  of  the  staff  interviewed  over  200  physical  scientists  at  two  basic  research  centers.  The 
interview  information  was  Kept  anonymous  to  protect  those  interviewed  and  to  encourage  frank 
opinions  from  the  scientists.  The  interviews  were  nonstructured,  informal  discussions  which 
generally  focused  on  the  nature  of  scientific  contributions  in  the  laboratories  where  the  study 
was  being  conducted,  the  kinds  of  effective  scientists  one  might  find  in  such  laboratories,  per¬ 
sonal  background  information,  and  oth^r  topics  brought  into  the  discussion  by  the  scientists, 
such  as  comments  about  their  working  conditions.  The  average  length  of  the  interviews  was  two 
hours.  Most  of  the  interviews  pursued  the  following  issues:  (a)  types  of  scientists,  (b)  kinds 
of  work  done,  (c)  personal  background  information,  (d)  probable  criteria  of  scientific  productivity 
and  creativity,  (e)  suggested  ways  for  recording  and  scoring  behavior  on  such  criteria,  (f)  nature 
of  the  talent  in  one's  own  laboratory,  (g)  ways  of  predicting  success  in  science,  (h)  conditions 
which  influence  scientific  contributions,  and  (i)  other  guestions  which  seemed  appropriate  in  each 
interview  situation. 

At  first  the  interviews  were  tape  recorded,  but  the  method  had  so  many  negative  features 
that  it  was  decided  to  merely  abstract  the  interviews  in  written  form  while  the  scientist  and 
interviewer  were  talking.  Many  scientists  objected  to  having  their  comments  appear  on  tape  and 
they  stated  that  they  did  not  feel  free  to  express  their  true  opinions  while  the  tape  recorder  was 
running.  Also,  the  available  recorder  was  cumbersome  to  transport  to  the  widely  scattered  offices 
and  interfered  with  the  smooth  flow  of  interview  information. 
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While  being  interviewed,  each  scientist  was  asked  to  discuss  freely  any  ideas  he  wished 
which  were  related  to  proauctivity  and  creativity  in  research,  e.g.,  passible  criteria  of  success 
in  science  and  possible  predictor  tests.  This  task  resulted  in  many  statements  which  implied 
that  working  conditions  have  important  effects  on  the  quantity  and  quality  of  scientists'  contribu¬ 
tions  as  well  as  statements  related  to  the  primary  goals  of  the  project. 

Information  about  the  effects  of  different  working  conditions  on  the  quantity  and  quality  of 
scientists'  contributions  is  needed  before  maximum  utilization  of  scientific  talent  can  truly  be 
achieved.  The  obvious  assumption  underlying  such  a  statement  is  that  certain  factors  found  in 
the  working  environments  of  scientists  affect,  either  favorably  or  unfavorably,  certain  aspects 
af  their  professional  performance. 

An  analysis  of  the  interview  statements  suggested  the  following  kinds  of  criteria: 

1.  Ratings  by  others  af  the  scientist  and  of  his  scientific  products.  Typical  of  such  cri¬ 
teria  were  subjective  ratings  af  productivity  by  one's  supervisors,  peers,  and  a  research  psycholo¬ 
gist;  a  complex  quality  and  quantity  evaluation  af  publications  and  research  reports;  and  the 
collection  of  quality  and  quantity  information  about  patents  and  invention  disclosures. 

2.  Collection  af  objective  facts  about  each  scientist  from  Air  Force  records.  The  number 
of  publications  one  had  produced  during  a  given  two-year  period  with  the  organization;  the  number 
of  patents,  invention  disclosures,  and  evaluations  of  them;  number  of  contracts  monitored;  and 
organizational  background  history  of  each  scientist  were  the  major  kinds  of  probable  criterion 
information  available  from  these  records. 

This  interview  information  was  utilized  quite  fully  in  the  criterion  study.  Likewise  all  the 
suggestions  in  the  interviews  about  characteristics  of  effective  scientists  were  considered  in 
planning  the  validation  study. 


THE  CRITERION  STUDY  ON  THE  MULTIPLE  CONTRIBUTIONS  OF  SCIENTISTS 

The  interviews  had  been  conducted  on  samples  af  scientists  at  twa  Air  Farce  research 
centers.  The  criterion  data  were  collected  an  a  sample  of  215  physical  scientists  working  in 
15  different  laboratories  at  one  of  these  centers.  Criteria  were  collected  on  17  instruments  yield¬ 
ing  a  potential  150  scares.  These  criteria  were  collected  fram  supervisors,  laboratory  chiefs, 
peers,  official  records,  reports  and  publications,  and  fram  the  scientists  themselves  (on  their 
society  membership,  etc.).  An  attempt  was  made  to  collect  criterion  data  an  all  of  the  dimensions 
thought  ta  be  important  as  a  result  af  the  interviews.  Same  necessary  control  data  such  as  age, 
years  of  experience,  education,  as  well  as  some  predictive  self-ratings,  were  also  collected  on 
this  sample.  The  subjective  ratings  at  the  scientists  and  their  products  were  extremely  time 
consuming,  and  the  raters  //ere  so  busy  that  individual  time  allowances  had  to  be  made  in  each 
case.  Some  scientists  were  reluctant  to  make  any  written  record  of  their  own  judgments  of  fellow 
scientists.  In  other  wards,  various  difficulties  were  encountered  in  getting  all  af  the  criterion 
forms  filled  out.  This  made  it  impassible  to  collect  the  data  in  a  short  period  of  time.  In  some 
cases,  negative  attitudes  of  certain  scientists  in  the  sample  had  to  be  overcome  by  intensive 
explanations  of  the  ultimate  use  af  the  data,  or  by  changing  the  methods  of  data  collection  to  fit 
their  specific  demands.  One  member  of  the  staff  remained  on  the  scene  during  the  eight-month 
period  of  the  data  collection  ta  insure  that  maximum  return  of  the  data  would  be  obtained. 

During  the  data  collection  phase  of  the  project  the  field  research  psychologist  continually 
received  feedback  from  the  scientists  about  the  ways  to  score  the  criterion  data,  together  with 
further  information  an  the  nature  of  scientific  productivity  and  creativity. 
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Several  members  of  the  initial  sample  were  found  to  lack  a  number  of  the  criterion  scores, 
and  49  scientists  were  eliminated  from  the  sample  prior  to  the  data  analysis  because  af  insufficient 
data.  Those  who  had  only  a  few  criterion  scores  missing  were  retained  in  the  sample  by  using  an 
estimated  score  for  each  of  the  missing  criterion  scores.  In  several  instances,  the  mean  af  the 
group  was  used  for  the  estimated  score,  but  occasionally  the  estimated  score  was  determined  from 
regression  results  found  in  small  pilot  studies  and  by  using  an  estimated  score  based  upon  the 
results  an  the  variable  that  was  most  like  the  missinq  ane.  Though  this  procedure  was  rather 
arbitrary  and  subjective,  it  allowed  a  relatively  large  sample  to  be  studied  by  estimating  a  few 
scores  an  these  difficult-to-obtain  criteria. 

The  characteristics  of  the  sample  as  ta  major  discipline  and  grade  were  as  follows.  By 
discipline,  35%  were  physicists;  7%  mathematicians;  48%  electronic  engineers;  and  10%  chemists. 
By  civilian  ar  military  grade,  2%,  were  GS  7;  ll?/o  GS  9;  18%  GS  11;  22%  GS  12;  26%  GS  13;  13% 

GS  14;  4%  GS  15;  2%  2nd  lieutenant;  and  2%  1st  lieutenant. 

ANALYSES  OF  THE  CONTRIBUTIONS  OF  SCIENTISTS3 

The  150  scores  originally  proposed  were  reduced  to  52  criterion  scores  for  this  first  study. 
This  reduction  in  scores  was  accomplished  as  a  result  of  several  small  pilot  studies  on  the 
data  which  led  to  combinations  af  hiqhly  related  scores  and  elimination  of  some  of  the  potential 
scores  for  reasons  that  seemed  justifiable.  For  example,  supervisory  ratings  an  17  characteristics 
were  reduced,  through  combining  scores,  to  10  ratings.  The  52  scores  included  11  ratings  and 
rankings  by  immediate  supervisors;  4  ratings  from  higher  level  supervisors,  the  laboratory  chiefs; 

6  ratings,  rankings,  and  nominations  by  peers;  9  scores  taken  from  official  records;  12  quantity 
and  quality  scores  on  research  reports  and  publications;  1  score  on  membership  in  professional 
societies;  4  ratings  fram  the  researchers  who  worked  on  this  project;  and  5  control  variables. 

These  control  variables  (such  as  age,  years  of  experience)  were  variables  which  were  not  directly 
concerned  with  contributions,  as  such,  but  which  might  account  for  some  of  the  differences  in 
contributions  among  scientists,  il  a  sizable  relationship  were  found  to  exist  between  any  control 
variable  and  a  particular  contribution  score,  it  would  be  necessary  to  partial  out  the  effects  of 
the  control  variable  in  the  contribution  score.  The  titles  af  each  of  these  52  scores  with  the 
source  from  which  each  ar.e  was  obtained  and  the  means  and  the  standard  deviations  are  listed 
in  Table  2,  Appendix  A.  The  correlations  cmong  these  scores  are  presented  in  Table  3,  Appendix 
A. 

Eight  different  sources  were  used  for  obtaininq  data  about  the  contributions  of  the  sample 
of  166  scientists  in  one  large  research  laboratory.  A  first  correlational  analysis  of  the  relation¬ 
ships  among  these  contribution  scores  was  made  in  terms  of  the  different  sources  of  information 
from  which  these  scares  were  obtained.  Scores  from  supervisors  usually  had  a  low  positive  cor¬ 
relation  with  those  obtained  from  peers.  Little  relationship  was  found  between  the  scores 
obtained  from  peers  and  those  from  laboratory  chiefs.  The  relations  between  subjectively- 
obtained  scores  and  scores  obtained  from  records  were  generally  negligible.  Scores  either  from 
supervisors  or  from  peers  generally  were  unrelated  to  scares  on  research  reports  and  publications, 
although  there  were  a  few  exceptions.  The  five  control  variables  were  age,  number  of  years  of 
education,  total  number  of  years  experience,  length  of  experience  at  this  particular  laboratory, 
and  a  score  on  freedom  from  publication  hindrances.  These  five  control  variables  were  generally 
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quite  independent  from  the  contribution  scores.  This  is  a  noteworthy  finding,  since  it  was  not 
necessary  in  this  study  to  partial  out  the  effects  of  any  of  these  five  variables  on  any  of  the 
contributions  scores  of  the  scientists.  For  example,  the  control  variable  of  total  number  of  years 
of  experience  accounted  for  little  or  none  of  the  individual  differences  in  contributions  of  these 
scientists.  However,  it  should  be  mentioned  that  the  effect  of  these  control  variables  had  already 
been  largely  cancelled  out  in  a  few  contributions  scores,  such  as  patent  rate  and  promotion  rate 
(Table  2,  items  28,  30).  Nonetheless,  it  is  interesting  to  find  that  the  creative  and  productive 
contributions  typically  did  not  vary  with  either  the  age  or  the  number  of  years  of  education  of 
this  sample  of  scientists. 

One  general  conclusion  drawn  from  this  first  analysis  is  that  the  data  on  contributions  of 
scientists  tend  to  be  independent  (much  more  often  than  not)  when  obtained  from  different  sources. 
The  second  overall  observation  is  that  although  the  typical  relationship  across  different  sources 
of  information  is  either  low  or  negligible,  a  wide  range  of  relationships  was  often  found  between 
different  pairs  of  scores  from  any  two  sources  of  information.  This  latter  finding  suggests  that  a 
better  basis  for  clustering  or  categorizing  these  contribution  scores  should  be  sought.  Admittedly, 
however,  the  source  of  information  does  account  for  some  of  the  variability  found  in  these  contri- 
bution  scores,  so  that  it  represents  an  important  aspect  in  the  total  problem  of  analyzing  the 
contributions  of  scientists. 

Another  approach  for  viewinq  the  relationships  among  the  contribution  scores  was  to  find 
the  number  of  other  scores  to  which  each  one  was  significantly  related.  At  one  extreme,  it  was 
found  lhat  one  score  of  contributions  was  related  to  only  a  single  other  contribution  score;  where¬ 
as  at  the  other  extreme  certain  contribution  scores  were  related  to  about  half  of  the  other  scores. 

On  the  average,  a  contribution  score  was  related  to  only  20%  of  the  other  scores.  This  finding 
does  illustrate  the  complexity  of  the  total  set  of  contributions,  since,  on  the  average,  four  out  of 
five  other  contribution  scores  are  independent  of  any  given  contribution  score. 

THE  MAIN  CATEGORIES  OF  THE  CONTRIBUTIONS  OF  SCIENTISTS 

The  next  method  used  to  process  the  data  was  the  factor  analysis  technique,  as  developed 
by  Thurstone.  In  the  factor  analysis  of  the  correlation  table  with  its  1326  correlations,  27  factors 
were  extracted  before  the  level  recommended  by  Tucker  to  stop  extracting  factors  had  been  reached. 
A  rotational  solution  using  Thurstone's  analytical  method  (1953)  has  been  finished  on  the  computer, 
based  upon  the  19  most  significant  factors,  although  only  15  categories  (criterion  and  control 
factors)  were  interpreted  and  retained  for  use  in  the  validation  study.  The  unrotated  factor  matrix 
and  the  transformation  matrix  have  been  included  as  Tables  4  and  5  in  Appendix  A.4  In  effect,  the 
number  of  main  categories  (factors  or  dimensions)  of  contributions  was  determined  from  the  rela¬ 
tionships  and  clustering  of  the  contribution  scores.  Each  score  of  the  contribution  of  scientists 
was  found  to  fall  significantly  into  one  or  more  categories  to  which  the  evidence  showed  it 
belonged.  This  technique  of  determining  the  main  categories  and  of  classifying  each  contribution 
score  into  its  appropriate  category  or  categories  was  accomplished  entirely  on  a  computing  ma¬ 
chine.  These  main  categories  proved  to  be  largely  unrelated  to  each  other  and  thus  formed  a  set 
of  15  relatively  independent  categories  into  which  the  contribution  scores  were  classified.  The 
results  show  that  the  source  of  criterion  information  is  certainly  not  the  sole  basis  of  difference 
among  tuese  factors  and  that  the  15  categories  provide  a  clear  and  sound  basis  for  organizing  the 
correlational  results  among  the  criterion  scores. 


A  secand-arder  factor  study  among  the  pnmory  factors  af  the  first-order  study  has  been  partly  completed,  as  a 
result  of  o  suggestion  from  a  leading  factor  analyst  who  hoped  that  qreater  simplicity  m  the  tatol  criterion  problem 
would  be  found  in  the  second-order  solution. 
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The  complexity  of  the  total  problem  is  again  indicated  fust  by  the  large  number  of  different 
categories  (15)  required  for  adequately  categorizing  the  52  contribution  scores;  and  second,  by 
no  mere  than  13  scores  sorting  into  any  single  category,  with  the  median  number  of  scores  per 
factor  about  six  out  of  a  possible  52.  The  machine  solution  indicated  that  two  particular  scores, 
number  supervised  and  the  productivity  rankinq  made  by  the  supervisor,  were  classified  into  a 
maximum  (in  this  problem)  of  four  out  of  15  categories,  6  scores  fell  into  three  categories  each, 

20  scores  were  classified  into  two  categories  each,  and  the  remaining  24  scores  each  fell  into 
only  one  of  the  15  different  categories.  This  sorting  of  the  various  contributions  of  scientists 
into  several  main  categories  provides  a  powerful  and  interesting  analytical  view  of  the  total 
problem  under  investigation. 

The  15  categories  of  contributions  have  been  arranged  according  to  whether  they  represent 
science-oriented  or  organizationally-oriented  behaviors.  The  first  categories  tend  to  be  science 
oriented  whereas  the  last  ones  (except  for  the  control  score  category)  are  more  organizationally 
oriented.  A  few  categories  in  the  middle  of  the  listing  are  ambiguous  and  are  not  easily  classified 
clearly  into  one  brrhese  two  broad  types.  The  appearance  of  scores  with  negative  weights  in 
some  of  these  categories  invariably  proves  to  be  an  interesting  and  provocative  phenomenon. 

Table  1  lists  each  contribution  score  as  it  was  classified  into  each  category,  together  with  its 
veight  in  that  category.  This  is  the  rotated  factor  matrix  with  rows  rearranged  and  with  all  load¬ 
ings  be  /  .20  omitted  in  order  to  display  the  factor  pattern. 

Brief  descriptions  of  the  15  relatively  separate  categories  are  presented  below,  with  some 
mention  cf  the  contribution  scores  that  fell  into  each  category.  The  scores  are  usually  presented 
so  that  the  ones  with  greatest  weiqnt  in  the  category  appear  first  in  the  description. 

Category  A.  This  category  represents  productivity  in  written  scientific  work.  A  good 
measure  of  this  cateqory  would  identify  scientists  who  are  effective  paper  workers  in  the  prepara¬ 
tion  of  written  articles,  reports,  technical  memos,  patent  applications,  and  other  scientific  papers. 
Such  persons  are  less  frustrated  than  the  average  scientific  worker  in  terms  of  the  amount  of  their 
scientific  work  which  is  completed  except  for  the  final  write-up.  However,  perhaps  from  their 
scientific  orientation  involving  their  dedication  to  completing  their  scientific  contributions  through 
the  final  write-up  staqe,  they  are  more  frustrating  to  their  supervisors  who  rate  them  low  on  cooper¬ 
ation.  The  other  two  scores  with  smaller  weights  in  this  cateqory  indicate  that  the  predictions 
about  future  productivity  are  above  average  for  these  productive  paper  workers  and  that  these 
persons  have  more  educational  background  in  sheer  number  of  years  than  do  the  typical  research 
workers  in  this  organization. 

Category  B.  This  category  seems  to  be  somewhat  complex  and  puzzling.  It  may  indicate 
the  degree  to  which  a  scientist  has  recently  produced  a  quantity  o /  research  reports  in  the  organi¬ 
zation,  though  it  was  not  particularly  related  to  total  productivity.  A  scientist  high  in  this 
cateqory  estimates  that  he  will  continue  to  be  productive  in  publishing,  has  fewer  than  average 
complaints  about  hindrances  to  publishing,  and  is  rated  slightly  above  average  on  drive,  resource¬ 
fulness,  and  persistence.  He  is  also  likely  to  engaqe  in  monitoring  work.  This  last  result  does 
not  seem  to  fit  unless  he  was  rewarded  (if  this  is  truly  a  reward)  for  his  production  of  research 
reports  by  being  qiven  more  monitorinq  work  or  unless  he  somehow  got  credit  for  research  reports 
due  to  his  monitorinq  role. 

Category  C.  In  general,  this  cateqory  includes  all  of  the  qualify  ratings  of  research  reports 
except  for  the  originality  rating.  The  originality  rating  was  the  most  different  of  the  five  ratinqs 
of  research  reports,  correlating  zero  with  the  rating  on  elegance.  Therefore  this  category  con¬ 
tains  the  judgment  by  experts  in  his  field  of  the  quality  of  a  scientist's  research  reports,  wherein 
some  elements  of  significance,  but  not  of  novelty,  are  included  in  the  concept  of  quality. 
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Score 

TABLE  1. 

Co ntr i  but  ion 

Factor 

Ana 

lysis 

of  Contributions  of  One  Sample  of  Scientists 

Categories  (or  Foctors) 

Nr  of 

,  Si  nni  f  i  f* hr ♦ 

«/i  y ii i  i  i  l u i <  I 

Nr 

Score  a 

A_ 

B 

D 

E_ 

H_  ±  J_ 

K 

L  M  N 

0  Loadings 

37 

Articles 

58 

-26  22 

3 

38 

Reports,  memos 

37 

1 

10 

Cooperotion 

-36 

64 

2 

28 

Potent  rote 

29 

21 

24 

3 

17 

Prod,  predicted 

27 

40 

2 

50 

Educ.  yrs. 

27 

23 

2 

41 

Write  up  incompl. 

-26 

1 

40 

Popers  completed 

21 

1 

22 

Re po rts  —  2  yrs 

62 

23 

2 

42 

Pub.  est. 

23 

1 

29 

Monitor 

22 

23 

59 

3 

2 

Drive  ,  re  source 

21 

63 

2 

43 

Free-pub.  bind. 

21 

-3  0 

2 

9 

Persistence 

20 

60 

2 

25 

Orgon.  reports 

70 

1 

24 

Relev.,  reports 

62 

1 

27 

Eleg.t  reports 

54 

1 

23 

Signif.,  reports 

42 

51 

2 

4 

Integrity 

2  3 

48 

2 

26 

Orig.,  reports 

72 

-21 

2 

31 

Suggestions 

21 

-24 

2 

36 

Societies 

53 

1 

35 

Activities 

22 

1 

34 

No.  supervised 

21 

34 

40 

24 

4 

52 

Months  in  center 

-20 

23 

48  3 

16 

Prod.  rnk.  peer 

61 

29 

2 

1 

Prod,  rnk,  super. 

58 

20  29 

25 

4 

45 

Consultont 

39 

1 

47 

Lab.  nomin. 

35 

20  49 

3 

44 

New  problems 

27 

23 

2 

12 

50%  retention 

23 

51 

2 

3 

Moth,  cogn, ,  etc. 

20 

54 

2 

1  5 

Sci.  chor.  cklist 

84 

1 

14 

Creot.  cklist 

79 

1 

1  3 

Prod,  cklist 

66 

20 

2 

8 

F  lexibility 

67 

1 

6 

Indep.  discovery 

65 

1 

5 

Desire  for  focti. 

54 

1 

11 

Creotion 

51 

1 

7 

Inform  .  obility 

50 

1 

19 

Likobility 

39 

1 

48 

Trustworthy 

-2  2 

1 

48 

Center  nomin. 

20  67 

21 

3 

39 

Popers  reod 

34 

38 

2 

21 

A  words 

25 

54 

2 

20 

Noncomply,  olter. 

-20 

1 

30 

Promotion  rote 

54 

-23  2 

51 

Work  yeors 

-2  3 

80  2 

32 

Poy 

89 

1 

33 

Primory  octivity 

81 

1 

18 

Proj.  coop. 

-28 

1 

49 

Age 

82  1 

Note.  —  Decimal  points  are  omitted. 
a  See  Appendix  A,  Table  2,  for  complete  names. 
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Category  D.  This  category  represents  oriqinality  o /  work  and  thought.  Such  original  thought 
apparently  is  carried  through  to  the  stage  where  it  appears  in  the  scientist's  research  reports 
judged  by  experts  in  his  field  to  be  original  and  significant,  in  patents,  and  in  suggestions  sub¬ 
mitted  and  officially  accepted  in  the  organization.  However,  organizational  awards  and  the 
creativity  ratings  made  by  the  supervisor  and  by  the  laboratory  chief  did  not  fall  into  this  category. 
Perhaps  this  category  is  really  limited  to  originality  in  one's  written  products.  It  is  interesting 
to  find  that  originality  is  guite  a  separate  characteristic  of  research  reports  as  distinct  from  high 
guality  ratings  of  elegance  and  organization  of  these  reports. 

Category  E.  The  number  and  importance  of  scientific  and  professional  societies  to  which  a 
person  belongs  is  the  main  feature  of  this  category.  Apparently  those  who  have  more  education, 
who  work  at  hiqher  level  activities  in  the  research  organization,  and  who  supervise  more  people 
are  qualified  to  belong  to  more  societies.  They  have  a  highei  self-rating  on  cognitive  abilities, 
but  are  not  the  workers  with  higher  seniority  at  this  research  organization. 

Category  F.  The  common  feature  appears  to  be  a  judgment  of  actual  work  output  or  produc¬ 
tivity  of  the  scientist  as  seen  by  those  who  work  near  him.  These  judgments  in  the  form  of  rank¬ 
ings,  nominations,  or  checklist  ratings,  are  made  by  the  peers  in  his  immediate  laboratory,  by 
his  immediate  supervisor,  and  by  the  chief  over  his  laboratory.  It  is  note-worthy  that  the  two 
hiqhest  weighted  scores  in  this  category  are  the  productivity  ranking  by  peers  and  the  productivity 
ranking  by  immediate  supervisors.  The  only  non-judgmental  score  in  this  category  was  the  patent 
rate  score.  Only  one  of  the  supervisor's  ratings  on  the  ten  rating  scales  appeared  with  a  signifi¬ 
cant  weight  even  though  the  supervisor's  productivity  ranking  of  the  scientists  was  the  next-to- 
highest  weighted  score  in  this  category. 

Category  G.  The  three  scores  in  this  category  were  all  obtained  from  laboratory  chiefs. 

They  clustered  together  more  highly  than  any  other  set  of  contribution  scores  in  the  study  and  all 
three  were  derived  from  successive  sections  of  checklist  items  that  appeared  on  the  checklist 
form  in  the  study.  On  this  form  the  first  two  checklists  on  creativity  and  ptoductivity  were  those 
developed  and  used  by  D.W.  Taylor  (1958),  pp.  36-43).  The  third  checklist  of  creative  character¬ 
istics  of  scientists  clustered  closely,  although  it  was  scored  by  merely  adding  the  number  of 
checks  instead  of  applying  predetermined  scale  values  to  each  item  checked.  This  category 
might  be  judged  to  represent  the  view  of  each  scientist  as  seen  from  certain  higher  levels  in  the 
organization. 

Category  H.  This  category  appears  to  be  the  supervisor's  overall  evaluation  of  the 
scientist  based  on  all  characteristics  of  his  job,  as  represented  in  the  10  rating-scale  scores 
used  in  this  study.  The  supervisor's  ranking  of  the  scientists  on  productivity  was  barely 
classified  into  this  category.  Nominations  by  peers  of  the  person  to  whom  they  would  most 
prefer  to  assign  a  new  problem  also  appeared  with  a  low  positive  weight.  The  number  of  journal 
articles  of  the  scientist  had  a  low  but  significant  negative  weight.  This  negative  weight  may 
again  be  indicative  and  supportive  of  the  conflicts  that  a  scientist  may  face  as  he  tries,  on  the 
one  hand,  to  satisfy  the  supervisory  organization  and,  on  the  other  hand,  to  complete  his  contribu¬ 
tion  to  his  scientific  field  by  publishing  articles  on  his  research  work. 

Category  /.  This  category  is  interpreted  as  likableness  as  a  worker  in  the  organization. 
The  scores  with  the  highest  weights  indicate,  at  least  indirectly,  how  well  various  persons  like 
to  work  with  him  and  to  what  degree  they  would  like  to  keep  him  as- a  fellow  worker.  These 
judgmental  scores  come  from  the  chiefs,  supervisors,  peers,  and  the  project  researcher  who 
interviewed  intensively  most  of  the  sample  and  also  worked  with  them  in  the  collection  of  the 
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contribution  scores.  It  is  interesting  that  the  number  of  journal  articles  appears  in  this  category. 
However,  the  nomination  by  peers  of  persons  whose  word  in  their  technical  field  can  be  highly 
trusted  has  some  component  within  it  that  is  negatively  related  to  this  likableness  characteristic  — 
this  negative  component  may  indicate  a  below  average  chance  of  scientists  so  nominated  being 
retained  if  layoffs  were  to  occur. 

Category  /.  The  most  common  thread  present  in  this  category  is  the  visibility  of  the  person 
or  of  his  name  throughout  the  organization  —  in  other  words,  how  well  known  is  he  in  person  or  by 
name  in  the  entire  organization.  He  tends  to  be  nominated  as  an  outstanding  scientist  by  persons 
outside  of  his  own  immediate  laboratory;  he  supervises  more  persons  than  the  average;  he  appears 
in  public  and  in  the  news  by  presenting  papers  (but  not  necessarily  publishing  them);  he  has 
worked  longer  with  the  organization;  and  he  has  somehow  managed  to  win  a  few  more  awards,  on 
the  average.  Being  so  visible  and  outwardly  active  in  supervising  and  in  presenting  papers,  he 
had  more  than  an  average  number  of  complaints  about  publishing  difficulties,  but  was  slightly 
more  cooperative  and  compliant  in  fulfilling  the  demands  of  this  project  than  was  the  typical 
scientist. 

Category  K.  The  highest  weighted  score  in  this  category  is  the  number  of  official  awards 
received  in  the  organization,  a  form  of  organizational  recognition  for  one's  contributions  in  the 
organization.  Perhaps  the  category  really  involves  social  recognition  for  one's  visible  type  of 
contributions  in  working  with  people  and  in  being  an  effective  oral  communicator.  The  basis  on 
which  awards  are  actually  granted  may  well  deserve  closer  study  since  the  awards  score  is 
related  to  the  number  supervised  (do  organizational  awards  mainly  go  to  supervisors?)  and  to  the 
number  of  papers  read,  another  visibility  index.  The  real  challenge  that  emerges  in  this  category 
is  that  the  number  of  organizational  awards  is  slightly  negatively  related  to  the  rating  of  originality 
of  the  scientists'  research  reports. 

Category  L.  This  category  may  represent  the  degree  to  which  each  scientist  has  status- 
seeking,  "organizationai-man  tendencies.  The  promotion-rate  index  is  the  score  that  appeared 
most  prominently  in  this  category.  Most  interestingly,  this  analysis  shows  that  those  who  make 
official  suggestions  to  the  organization  and  who  rate  themselves  as  being  high  in  the  desire  to 
discover  new  principles  in  science  have  a  below-average  rate  of  promotion.  One  possible  expla¬ 
nation  is  that  those  with  a  slower  promotion  rate  tend  to  rationalize  by  claiming  that  they  have  an 
above  average  desire  to  discover  in  science.  An  alternate  explanation  is  that  some  research 
organizations  may  tend  to  promote  those  who  do  not  make  official  suggestions  as  well  as  those 
without  a  strong  desire  to  discover  (but  possibly  with  a  strong  desire  for  promotion  and  status). 
Further  evidence  relevant  to  these  and  other  possible  explanations  is  presented  later  in  the 
validation  results  for  this  category. 

Category  M.  This  category  seems  to  involve  the  current  organizafionai  status  of  each 
worker  in  the  sample.  His  official  status  in  the  organization  (which  is  consistent  with  the 
typical  American  view  of  status)  is  indicated  by  his  rate  of  pay,  the  index  of  the  level  of  his 
primary  activity,  and  the  number  of  workers  he  supervises. 

Category  N.  The  most  prominent  score  in  this  category  is  the  contract  monitoring  load  of 
each  scientist.  Apparently  those  who  are  ranked  high  on  productivity  by  their  supervisor  and 
who  recently  produced  a  large  number  of  research  reports  within  the  organization  tend  to  have  an 
above  average  load  of  monitoring  research  contracts  with  outside  contractors.  The  persons  with 
heavy  contract  monitoring  loads  usually  had  some  responsibilities  for  inside  research  work,  so 
that  they  were  extremely  busy  people  and  were  rated  by  our  project  researchers  as  being  below 
average  in  their  cooperation  on  our  research  project. 
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Category  0.  This  category,  which  includes  three  control  scores,  seems  to  measure  predomi¬ 
nantly  the  total  years  o /  work  experience  of  each  worker.  Age  is  highly  related  to  years  of  work 
experience,  and  the  amount  of  time  worked  ct  this  research  organization  is  a  part  of  the  total 
time  a  scientist  has  worked  at  all  organizations.  The  only  other  score  in  this  category  was  pro¬ 
motion  rate,  with  a  small  negative  weight. 

The  other  main  control  scores  of  years  of  education  and  freedom  from  publication  hindrances 
did  not  appecr  in  this  category.  In  fact,  the  number  of  years  of  education  appeared  only  in  Cate¬ 
gory  A  involving  the  ability  to  do  effective  paper  work  and  in  Category  E  involving  scientific  and 
professional  society  membership. 

Six  of  the  above  15  categories  contained  at  least  one  score  that  might  be  called  a  creativity 
score.  For  example,  Categories  A  and  F  are  complex  and  include  patent  rate  within  them;  Cate¬ 
gory  D  involves  originality  in  written  work:  Category  G  includes  a  creativity  rating  by  the  labora¬ 
tory  chief;  Category  H  includes  a  supervisory  rating  on  creativity  as  well  as  on  flexibility  and 
independence;  and  Category  K  may  entail  some  form  of  creativity  in  administrative  work  since 

the  score  on  organizational  awards  appears  most  strongly  in  it.  ^ 

Initially  we  were  somewhat  puzzled  about  finding  such  great  complexity  in  the  total  contribu-  I 

tion  area,  as  indicated  by  the  many  low  intercorrelations  (low  overlap)  and  thus  the  high  specificity 
in  these  measures  on  contributions.  As  time  has  elapsed,  however,  we  are  getting  indications 
from  other  studies  that  this  finding  of  relatively  great  complexity  is  quite  sound. 

Admittedly  this  study  has  involved  analysis,  but  not  synthesis.  No  attempt  has  been  made 
to  estimate  the  total  contribution  of  a  scientist  by  combining  in  some  best-weighted  fashion  his 
contributions  in  each  of  the  above  relatively  separate  categories.  Thus  no  value  judgment  has 
been  made  about  the  relative  importance  of  each  ot  these  categories,  though  any  organization 
could  make  such  judgments  in  terms  of  its  particular  overall  qoals. 

THE  VALIDATION  STUDY  TO  IDENTIFY  CHARACTERISTICS  OF  SCIENTISTS 
RELATED  TO  EACH  TYPE  OF  CONTRIBUTION 

THE  CRITERIA  AND  CONTROL  SCORES 

The  second  major  study  required  in  the  project  was  to  determine  each  set  of  psychological 
characteristics  related  to  each  of  the  different  categories  of  contributions  of  scientists.  In  effect, 
there  were  17  different  tarqets  (criterion  or  cateqory  scores  of  contributions  of  scientists)  used  in  j 

this  second  study.  A  variety  of  psychological  tests  yielding  a  large  number  of  scores  were  cor¬ 
related  with  each  of  these  17  criterion  scores  to  identify  the  characteristics  that  correlated  signifi¬ 
cantly  with  each  type  of  contribution  in  science. 

The  14  categories  for  which  criterion  scores  were  computed  (numbered  1  to  14  in  this  valida¬ 
tion  study)  are  listed  below.  The  score  for  each  criterion  category  was  obtained  by  a  best-weighted 
combination  of  the  set  of  original  criterion  scores  with  the  highest  loadings  in  that  category.  The 
weights  were  obtained  by  means  of  a  multiple  correlational  analysis  aqainst  the  total  factor  score 
for  each  criterion.  (See  Table  6  in  Appendix  A.)  Three  original  scores  were  retained  as  separate 
criterion  scores  15,  16,  and  17,  since  they  represent  typical  criteria  used  in  studies  on  the  job. 

These  were  a  peer  ranking  on  overall  productivity,  a  combined  supervisory  rating  on  drive  and 
resourcefulness,  and  a  supervisory  rating  on  creativity.  Table  7,  Appendix  A,  presents  the  inter- 
correlations,  means,  and  standard  deviations  for  the  17  criterion  scores. 
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The  last  category  in  the  criterion  study  contained  the  three  main  control  scores:  age,  total 
years  of  work  experience,  and  total  months  of  work  experience  at  this  research  center.  These 
three  control  scores,  though  highly  related,  were  retained  as  three  separate  control  scores,  num¬ 
bered  18,  19,  and  20  in  the  validation  study 

Another  control  score,  years  of  education,  was  found  in  the  criterion  study  to  be  related  to 
only  two  categories,  with  a  loading  of  .27  in  the  category  of  productivity  in  written  scientific  work 
(effectiveness  as  a  paper  worker)  and  .23  in  the  category  of  scientific  and  professional  society 
membership  (for  which  there  may  be  educational  prerequisites).  Since  years  of  education  can  be 
alternately  considered  to  be  a  predictor  score  as  used  in  many  selection  programs  and  since  its 
validities  had  been  determined  and  were  found  to  be  essentially  zero  for  12  of  the  14  criterion 
factors,  it  was  omitted  as  a  score  in  the  validation  study,  c 

SEVENTEEN  MEASURES  OF  CONTRIBUTIONS  OF  SCIENTISTS 

1.  Productivity  in  written  work  (effectiveness  in  completing  paper  work). 

2.  Recent  quantity  of  research  reports  (number  of  articles,  research  reports,  etc.,  in  a  two-year 
period). 

3.  Quality  (without  originality)  of  research  reports. 

4.  Originality  of  written  work. 

5.  Scientific  and  professional  society  membership. 

6.  Actual  quantity  of  work  output  as  judged  by  peers,  supervisors,  and  laboratory  chiefs  (hiqher 
level  supervisors) 

7.  Creativity  rating  by  laboratory  chiefs  (higher  level  supervisors). 

8.  Overall  performance  (quality  ratings  by  supervisors  on  10  scales). 

9.  Likableness  as  an  effective  member  of  the  research  group. 

10.  Visibility  of  the  scientist  (well  known  by  person  or  by  name). 

11.  Recognition  for  organizational  contributions  (organizational  awards). 

12.  Status-seeking,  "organizational-man"  tendencies. 

13.  Current  organizational  status. 

14.  Contract  monitoring  load. 

15.  Peer  ranking  on  productivity  as  a  scientist. 

16.  Supervisory  rating  of  drive-resourcefulness. 

17.  Supervisory  rating  of  creativity. 

THREE  CONTROL  SCORES 

18.  Age. 

19.  Total  years  of  work  experience. 

20.  Total  months  of  work  experience  in  the  research  center  studied. 

The  question  on  these  three  control  scores  is  whether  age,  total  work  experience,  and  total 
experience  in  this  research  setting  are  related  to  any  of  the  above  17  contributions  or  to  any  of 
the  psychological  characteristics  leading  to  the  above  contributions. 


An  additional  factor  emerqed  primarily  among  tie  self-ratings  in  the  first  study.  Although  self-ratings  hod  been 
availoble  eorly  ond  four  self-rating  scores  had  been  included,  perhaps  prematurely,  in  the  criterion  study,  these  self- 
ratings  were  considered  to  be  predictors  rather  than  criterion  scores;  so  they  were  generally  not  reported  in  the  criterion 
study  and  were  included  separately  among  the  predictor  scores  in  the  validation  study. 
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SELECTION  AND  DEVELOPMENT  OF  PSYCHOLOGICAL  TFST  SCORES 


The  problem  of  the  selection  ond  development  of  tests  was  especially  difficult  because  of 
the  numerous  possibilities  to  consider.  The  research  team  decided  to  use  promising  scores  from 
intellectual,  self-rating,  personality,  motivational,  and  biographical  tests;  so  o  variety  of  tests 
of  each  type  were  considered  for  inclusion.  Multiple  scores  per  test  were  favored  wherever  they 
were  promising  ond  possible,  ond  no  limitation  of  using  only  existing  tests  was  imposed. 

Suggestions  for  tests  or  test  ideos  were  found  from  the  interviews  of  the  scientists,  from 
intensive  studies  of  the  three  Utah  creativity  conference  reports  (which  yielded  mony  more  than 
one  hundred  suggestions  including  several  with  some  supportive  evidence  from  smoller  studies), 
from  basic  reseorch  on  intellectual  and  nonintellectuol  chorocteristics ,  including  research  on 
communication  abilities  (C.W.  Taylor  ef  ah,  1958)  and  from  other  readings  and  research  experi¬ 
ences  of  the  investigators.  Toble  8,  Appendix  A,  gives  the  list  of  names  ond  numbers  of  the 
130  scores  to  be  volidoted  os  predictors. 

Direct  self-ratings  on  17  chorocteristics  suggested  by  the  scientists  were  collected  on 
speciolly  constructed  seporate  roting  sheets  durinq  the  first  criterion-study  field  trip.  These  17 
characteristics  were  drive,  mathematical  ability,  resourcefulness,  cognition,  integrity,  desire-for- 
focts,  desire-for-principles,  desire-for-discovery ,  informative  obility,  skill  in  design,  flexibility, 
persistence,  independence,  discrimination  of  value,  cooperotion,  intuition,  ond  creativity. 

After  consideration  of  several  competing  devices,  the  personality  test  chosen  wos  the 
Personality  Research  Inventory,  recently  developed  by  Saunders  (1955)  in  his  personality  reseorch 
program  at  Educational  Testing  Service,  which  yielded  26  scores  with  verbol  descriptions  similar 
to  those  felt  to  be  important  in  creative  scientists.  The  26  PRI  scores  were  self-insight,  free- 
floating  anxiety,  self-acceptance,  tolerotion  of  frustration,  toleration  of  ambiguity  or  complexity, 
compulsiveness,  impulsiveness,  altruism,  tolkativeness,  self-sufficiency,  gregoriousness,  oggres- 
siveness,  attitude  to  work,  foresight,  belief  in  right  of  individuals,  belief  in  rights  of  groups, 
sociol  conscience,  stotus  aspiration,  social  know-how,  social  status,  masculine  vigor,  artistic 
vs.  practical,  spiritual  vs.  material,  progressive  vs.  conservative,  liking  to  think,  ond  PRI 
acquiescence. 

More  than  40  intellectuol  tests  were  studied  after  being  judged  to  have  potential  volidity. 
Seven  short  and  factoriolly  different  intellectual  tests,  most  of  which  hod  multiple  scores,  were 
finally  chosen,  including  two  on  asking  questions  and  on  suggesting  improvements,  both  of  which 
seemed  to  have  some  motivational  ospects.  The  tests  were  Match  Problems  (Adaptive  Flexibility 
factor)  with  three  scores  on  correct  responses,  wrong  attempts,  and  per  cent  correct;  Consequences 
(Oriqinality  and  Broadly  Diffused  Attention  foctors)  with  two  scores  on  total  acceptoble  responses 
and  remote  responses;  Revision  II  (C.W.  Toylor  et  a L,  1958),  the  most  frequently  valid  predictor 
in  the  communication  abilities  project,  with  three  scores  on  ideas,  words,  and  rotio  of  words  per 
idea,  Visuol  Imagery6  (probably  Visualization  factor)  with  three  scores  on  correct  responses, 
wrong  attempts  plus  skips,  ond  per  cent  correctly  morked;  Word  Association  (Associational 
Fluency  factor)  with  three  scores  on  total  words,  totol  close  synonyms,  ond  number  of  sets; 
Pertinent  Questions  (Conceptual  Foresight  Foctor)  with  only  one  score,  the  number  of  suitoble 
responses;  and  Apparatus  Test  (Sensitivity  to  Problems  factor)  with  only  one  score,  the  number 
of  suggested  improvements.  7  The  last  two  tests  were  selected  because  these  were  intellectuol 
tests  that  were  motivotionol  in  nature. 


6  Revised  from  a  test  developed  by  Lloyd  Lofquist  ot  Camp  Hood,  Texas,  during  World  War  11. 

7 

All  tests  not  directly  cited  obove  were  developed  by  J.P.  Guilford  and  his  associates  at  the  University  at 
California.  They  are  described  in  the  five  reports  by  Guilford  et  al .  listed  in  the  Hr  fcrcncvs. 
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A  special  motivation  instrument  was  developed  as  a  result  of  our  previously  successful 
experience  with  a  similar  device  in  the  earlier  communications  abilities  project.  The  minimum 
aspiration  level  in  each  of  12  relevant  areas  was  measured  by  this  device  which  instructed  the 
scientist  to  indicate  the  minimum  level  with  which  he  would  be  satisfied  in  each  of  these  abilities 
and  skills  felt  to  be  needed  in  scientific  work:  reading  skills,  speaking  skills,  listening  skills, 
writing  skills,  guantity  of  work  output,  being  well  liked,  administrative  advancements,  being 
well  known,  quantity  of  reports,  theoretical  contributions,  experimental  contributions,  and  level 
of  original  work. 

Cattell's  new  Motivational  Analysis  Test8  (MAT,  which  has  also  been  called  the  Dynamic 
Analysis  Test)  with  its  10  scores  was  also  used.  The  10  scores  were  fear,  sex,  assertion,  self¬ 
sentiment,  wife-sweetheart,  sadism,  career,  super  ego,  parents-home,  narcism-ccmfort.  The 
strong  hope  was  that  at  least  some  of  the  scores  from  these  different  motivational  measures  would 
help  to  account  for  individual  differences  in  the  contributions  of  scientists. 

A  special  Creative  Process  Check  List  was  desiqned  from  the  many  descriptions  of  the 
creative  process,  with  the  hope  that  valid  scores  could  be  attained  from  checklist  responses  on 
one's  state  of  attention  and  one's  feelings  before,  during,  and  after  solving  problems  in  science. 

The  construction  and  scoring  of  this  instrument  is  presented  in  Appendix  B  as  an  illustration  of 
the  test  development  work  in  this  project. 

Each  scientist  was  "followed  back"  to  his  undergraduate  institutions  to  obtain  his  under¬ 
graduate  grade-point  average. 

A  300-item  biographical  inventory  was  constructed  from  a  large  pool  of  rough  items  already 
assembled  as  well  as  from  the  interview  suggestions,  from  the  biographical  results  reported  at 
the  Utah  creativity  conferences,  and  from  the  biographical  results  found  on  70  scientists  in  the 
master's  thesis  by  Robert  Ellison  (1960).  The  plan  was  to  try  out  both  empirical  and  a  priori 
keys  on  this  biographical  inventory,  using  both  subscores  and  total  scores  on  each  empirical  key. 
This  type  of  inventory  was  used  because  in  every  study  on  scientists  known  at  that  time,  selected 
biographical  items  had  produced  promising  results.  Separate  empirical  keys  were  developed  for 
each  of  the  criterion  scores  1,  4,  6,  8,  12,  and  16.  A  total  of  12  a  priori  biographical  scores  were 
used  which  were  judqed  by  their  content  to  measure  professional  self-confidence,  emotional 
restraint,  low  sociability,  high  self-sufficiency,  inner  directedness,  dedication  to  work,  liking 
to  think,  intellectual  thoroughness,  social  desirability,  self-reported  academic  level,  modal 
response,  and  general  activity  level  (see  Tab^e  8,  Appendix  A). 

An  interesting  side  study  was  made  possible  through  the  availability  of  four  sets  of  ratings 
across  the  same  17  characteristics  presented  in  the  list  of  self-ratings  above.  These  were  the 
supervisor's  ratings  of  the  scientist  and  of  the  scientist's  job  plus  the  scientist's  rating  of  himself 
and  his  job.  Six  matchings  of  profiles  were  accomplished  as  follows,  with  the  first  four  being 
much  more  meaningful  than  the  last  two:  (1)  the  supervisor's  ratings  of  both  the  scientist  and 
the  scientist's  job,  (2)  the  scientist's  rating  of  both  himself  and  his  jub,  (3)  both  the  supervisor's 
rating  and  the  scientist's  rating  of  the  scientist,  (4)  both  the  supervisor's  rating  and  the  scientist's 
rating  of  the  scientist's  job,  (5)  the  supervisor's  rating  of  the  scientist  and  the  scientist's  rating 
of  the  job,  and  (6)  the  scientist's  rating  of  himself  and  the  supervisor's  rating  of  the  scientist's 
job.  Three  different  types  of  profile-matching  scores  were  applied  to  each  of  the  six  matches 


8  Test  obtained  through  personal  communication  with  H.B.  Cottell,  1959. 
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above.  One  was  a  total  matching,  a  second  was  a  matching  with  only  the  scatter  and  shape  of 
profiles  allowed  to  vary  (with  differences  in  level  corrected  out),  and  a  third  with  only  the  shape 
allowed  to  vary  (with  differences  both  in  level  and  in  scatter  corrected  out.)  Thus,  18  different 
profile-matching  scores  were  available,  which  were  correlated  with  the  criterion  scores  1-20 
(including  the  last  three  control  scores  of  age,  total  years  of  work  experience,  and  total  months 
of  experience  at  this  research  center).  This  yielded  a  38  x  38  correlation  table  with  a  section  in 
it  which  contained  18  x  17  or  306  validity  coefficients  for  the  18  profile-matching  scores.  Four 
of  these  profile-matching  scores  were  selected  for  inclusion  in  the  final  validation  study.  These 
four  scores  had  a  sizable  number  of  significant  correlations  with  the  criteria  and  also  represented 
the  three  different  types  of  scoring  of  the  profiles.  They  were  the  total  profile-matching  score  on 
the  supervisor's  rating  of  both  the  scientist  and  the  scientist's  job;  the  shape-only  profile-matching 
score  on  the  same  two  sets  of  profiles;  the  total  profile-matching  score  on  both  the  supervisor's 
and  the  scientist's  ratings  of  the  scientist;  and  the  scatter-and-shape  profile-matching  score  of 
both  the  supervisor's  and  the  scientist's  rating  of  the  scientist's  job. 

In  summary,  a  total  of  130  scores  were  used  in  the  experimental  battery  to  measure  the  psy¬ 
chological  characteristics  of  the  scientists.  These  130  scores  were  derived  from  the  following 
16  instruments,  six  of  them  developed  on  this  project  and  most  of  the  others  relatively  new:  Self- 
Ratings  (17  scores),  Personality  Research  Inventory  (26  scores),  7  aptitude  tests  (16  scores),  a 
Minimum  Satisfactory  Level  scale  (12  scores),  a  Motivational  Analysis  Test  (10  scores),  a  Cre¬ 
ative  Process  Check  List  (2  scores),  Grade-Point  Average  (1  score),  Biographical  Information 
Blank  (42  scores),  and  two  rating  devices  yielding  two  sets  of  supervisory  ratings  of  the  scientist 
and  the  scientist's  job  plus  the  scientist's  rating  uf  himself  and  his  job  (from  which  ratings  the 
4  profile  matching  scores  were  obtained). 

the:  sample: 

The  main  sample  in  the  validation  study  consisted  of  107  physical  scientis.s  for  whom  com¬ 
plete  or  almost  complete  data  were  available  on  all  criterion  scores  and  all  predictor  scores.  We 
faced  a  dilemma  in  considering  whether  to  add  30  other  scientists  who  had  several  scores  missing, 
but  for  whom  a  large  majority  of  the  150  scores  were  available.  We  finally  decided  to  use  the 
smaller  of  107  scientists  with  practically  complete  sets  of  scores  instead  of  expanding  to  a  larger 
sample  of  137  scientists  through  estimating  many  missing  scores. 

Comparison  of  Turnover  with  Non-turnover  Scientists.  During  the  nearly  two  years  of  data 
collection  28  scientists  left  the  organization.  The  main  sample  who  remained  on  the  job  were 
compared  with  the  "turnover"  sample  on  the  final  set  of  criterion  scores  (but  not  on  the  psycho¬ 
logical  test  scores),  including  14  criterion  factors,  3  rankings  or  ratings  by  peers  and  immediate 
supervisors,  and  3  control  variables  of  age,  total  years  of  work  experience,  and  total  months  of 
experience  at  this  research  center.  The  significant  differences  (all  at  the  .01  level)  were  that 
the  turnover  sample  had  a  much  lighter  contract  monitoring  load,  were  younger,  and  had  less  total 
work  experience  than  the  main  sample  who  remained  with  the  research  center.  Some  slight  differ¬ 
ences  (which  were  not  significant  at  the  .05  level)  were  that  the  turnover  sample  had  less  work 
experience  at  this  research  center,  produced  slightly  higher  quality  and  also  slightly  more  original 
research  reports,  had  slightly  lower  organizational  status,  were  slightly  less  effective  paper 
workers,  were  judged  to  be  slightly  less  productive  in  total  work  output,  and  received  slightly 
higher  ratings  on  creativity  by  laboratory  chiefs.  (See  Table  9,  Appendix  A.) 

In  summary,  these  findinqs  were  generally  negative  since  only  3  out  of  20  comparisons 
showed  significant  differences.  The  other  differences  proved  to  he  nonsignificant,  but  did  give 
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some  further  suggestion  that  the  turnover  sample  had  some  favorable  as  well  as  some  unfavorable 
characteristics.  The  turnover  sample  wos  primarily  a  younger,  less  experienced  group,  not  as 
fully  established  in  the  research  center  organization  as  was  the  non-turnover  group  of  scientists. 

Comparison  oi  the  Fully  Cooperative  with  the  Other  Scientists  in  this  Project .  A  similar 
study  has  been  completed  on  the  question  of  whether  the  sample  of  107  scientists  who  cooperated 
fully  in  this  lengthy  research  project  were  either  representative  of  or  differed  in  any  major  respects 
from  the  49  scientists  who  for  various  reasons  were  unable  to  cooperate  completely,  although  they 
did  cooperate  considerably  since  only  a  very  few  predictor  scores  out  af  a  very  large  number  were 
lacking  on  most  of  these  less  cooperative  scientists.  Ma  iy  of  these  scientists  had  also  given 
considerable  time  and  help  to  this  project  during  the  interview  study  and  the  criterion  study  in 
which  we  obtained  nearly  complete  criterion  data  on  all  of  them. 

Again  the  comparisons  between  groups  were  made  on  only  the  final  set  of  criterion  scores, 
and  not  on  the  psychological  test  scores.  Those  who  cooperated  fully  were  found  (at  the  .01  level 
of  significance)  to  be  mare  effective  in  completing  paper  wark  than  those  who  did  nat  cooperate 
fully.  This  difference  is  reasonable  and  could  be  expected,  since  much  paper  work  was  required 
by  our  project.  The  cooperators  werealsa  found  (at  the  .05  level)  ta  have  less  work  experience 
at  this  research  center  and  to  have  a  higher  score  on  professional  society  memberships  than  the 
less  cooperative  group.  Slight,  though  nat  significant  differences  were  noted  on  other  criteria  as 
follows.  The  cooperators  had  received  slightly  less  recognition  for  organizational  contributions, 
were  rated  as  more  likable  as  members  of  the  research  team,  showed  slightly  more  status-seeking 
characteristics,  had  slightly  less  total  years  of  work  experience,  showed  slightly  more  originality 
of  written  work,  and  were  given  slightly  lawer  supervisory  ratings  on  drive  and  resourcefulness. 
(See  Table  10,  Appendix  A.) 

In  general,  the  cooperating  sample  was  not  particularly  different  from  the  less-cooperating 
sample  in  17  cut  of  20  bases  of  comparison,  differing  significantly  only  in  effectiveness  in 
completing  paper  wark,  in  total  work  experience  at  this  research  center,  and  in  total  score  on 
professional  society  memberships.  Consequently,  on  17  af  the  criteria  it  is  deemed  reasonable  ta 
consider  the  main  sample  of  scientists  to  be  representative  af  this  non-turnaver  population  af 
research  scientists.  If  proper  allowance  ar  adjust... ^-nt  is  made  on  the  main  sample  for  possible 
restriction  of  range  effects  on  the  other  three  criteria,  then  the  main  sample  could  be  considered 
as  representative  of  the  non-turnover  population  of  scientists  at  this  research  center  on  all  20 
criterion  characteristics. 

It  should  be  added  that  there  were  counterbalancing  trends  on  the  criteria  for  the  turnover 
sample,  in  comparison  with  the  less  cooperative  sample,  which  resulted  in  the  combined  sample 
of  those  scientists  not  studied  being  more  similar  ta  the  main  sample  studied.  For  example, 
those  who  resigned  were  significantly  younger  and  those  who  were  unable  to  cooperate  fully 
tended  to  be  slightly  older  than  the  main  sample.  Another  example  is  that  the  resignation  sample 
had  significantly  less  total  work  experience  than  the  main  sample,  whereas  the  less  cooperative 
sample  had  significantly  more  total  work  experience. 

COLLECTION  AND  PROCESSING  OF  TEST  DATA 

The  criterion  data  (scores  1  -20)  were  collected  during  the  first  extended  field  trip  during 
the  initial  study.  The  large  battery  of  test  scores  were  collected  by  means  af  four  additional 
short  field  trips,  spaced  across  several  months.  During  each  trip,  the  busy  scientists  were  asked 
to  take  tests  requiring  ane  or  more  hours  of  their  valuable  time.  Certain  tests  with  time  limits 
had  to  be  scheduled  for  testing  in  small  groups;  whereas  the  scientists  were  asked  to  complete 
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other  tests,  such  as  the  biographical  inventory,  durinq  time  that  they  could  find  available  at  work 
or  at  home.  Two  members  of  the  research  team  were  needed  on  the  later  field  trips  to  accomplish 
the  complicated  field  work.  The  last  field  trip  was  designed  solely  to  collect  missing  data  on 
scientists  who  had  either  been  away  on  a  trip,  absent  from  work,  or  too  husy  to  find  time  for 
testing  during  one  01  more  of  the  previous  trips.  Overall,  the  cooperation  was  excellent,  especially 
considering  the  many  hours  of  their  time  that  were  required  during  the  interviews,  the  long  criterion 
data  collection  period,  and  the  four  additional  field  trips  for  obtaining  their  scores  on  a  large 
variety  of  psychological  tests. 

Undergraduate  grade-point  averages  were  obtained  by  writing  to  the  college  or  university  and 
obtaining  this  averaqe  together  with  a  description  of  the  grading  system  and  their  method  of  con¬ 
verting  other  grading  systems  to  their  grading  system.  These  grades  for  each  scientist  were  then 
converted  to  the  one  most  common  system  found,  to  yield  the  qrade-point  average  used  in  the  study. 

The  intercorrelations  among  the  150  scores  on  the  sample  of  107  scientists  were  computed 
on  the  IBM  709.  This  correlation  table,  containing  11,175  different  Pearson  correlation  coefficients, 
can  be  viewed  as  having  four  main  subdivisions:  first,  a  section  of  136  different  intercorrelations 
among  the  17  criterion  scores  reproduced  as  Table  7,  Appendix  A;  second,  a  section  of  8,385  dif¬ 
ferent  intercorrelations  amonq  the  130  psychological  test  scores;  third,  the  444  correlations  of  the 
three  control  scores  with  all  variables;  and  fourth,  the  very  important  validity  section  showing  the 
2,210  validity  coefficients  for  the  130  psychological  test  scores  against  the  17  criterion  scores. 

In  the  remaining  pages,  the  writeup  will  focus  on  the  validity  section  of  the  correlation  table, 
since  the  intercorrelations  among  the  criteria  were  the  main  concern  in  the  previous  study  in  the 
project.  All  of  these  validity  coefficients  are  presented  in  Table  11,  Appendix  A.  The  intercor¬ 
relation  coefficients  among  the  130  test  scores  will  be  important  when  attempts  are  made  later  to 
form  best-weighted  combinations  of  these  scores  to  predict  each  set  of  criterion  scores. 

the  validation  results 

The  validity  coefficients  showed  the  relation  between  criteria  and  test  scores  collected  on 
present  employees  with  the  test  scores  collected  at  a  later  time  than  the  criteria.  The  collection 
of  predictor  test  data  began  approximately  nine  months  after  the  criterion  data  were  collected. 
Including  the  patch-up  trip,  there  were  four  different  data-collection  sessions  spaced  about  ft  ar 
months  apart.  Consequently,  this  study  design  entailing  the  testing-of-present-employees  yields 
"follow-back"  validities  more  than  concurrent  validities  or  follow-up  (predictive)  validities.  It 
should  be  added,  however,  that  most  of  the  so-called  concurrent  validity  studies  really  yield 
"follow-back"  validities,  if  the  sound  practice  is  followed  of  collecting  the  criterion  data  prior 
to  the  psychological  test  data.  In  such  cases,  though,  it  is  rare  to  have  a  time  interval  of  more 
than  a  few  hours  or  days  between  the  collection  of  the  criterion  data  and  the  test  data. 

Predictability  ot  the  Criteria.  The  general  finding  across  all  17  criteria  was  that  slightly 
more  than  one  out  of  four  scores  were  valid,  so  that  in  the  validity  section  of  the  correlation 
matrix,  there  were  568  significant  validity  coefficients  found  in  the  project.  These  significant 
validity  coefficients  ranged  from  .19  (at  the  .05  level)  to  the  high  .40's.  Although  the  individual 
coefficients  were  generally  not  very  high,  most  of  them  seemed  to  be  quite  meaningful  in  terms 
of  the  particular  on-the-job  criteria.  The  prediction  task  is  very  difficult  where  there  are  so 
many  different  criteria  of  performance  on  the  job,  and  where  there  can  be  some  restriction  on  both 
criterion  and  test  scores.  For  most  criteria,  a  large  number  of  individually  valid  scores  can  be 
combined  to  produce  a  battery  validity  with  a  noticeable  increase  in  validity  over  the  coefficient 
for  the  most  valid  test  score.  In  dealing  with  these  many  valid  scores,  one  gets  the  impression 
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of  beinq  in  a  situation  of  coinbininq  a  larqe  number  of  items  in  item  analysis  work  rather  than 
combininq  only  a  handful  of  test  scores  as  is  usually  done  in  forminq  test  batteries.  As  with 
items,  these  data  need  cross  validation  to  estimate  the  shrinkaqe  that  miqht  occur  in  new 
samples. 


The  followinq  list  presents  the  17  criteria  arranqed  from  hiqhest  to  lowest  in  terms  of  their 
predictability.  The  criteria  with  the  qreatest  percentaqe  of  significant  validities  are  at  the  top 
of  the  list  of  criteria,  which  are  presented  in  rank  order.  The  percentaqe  of  the  validity  coeffi¬ 
cients  found  to  be  siqnif icantly  qreater  than  zero  for  each  criterion  is  placed  in  the  parentheses. 
The  reader  should  be  warned  that  one  biasinq  factor  underlies  this  list;  through  item  analysis 
methods,  five  empirical  keys  were  developed  on  the  Biographical  Information  Blank  for  each  of 


the  six  criteria  marked  with  an  asterisk. 

9.  Likableness  as  a  Research  Team 
Member  (44%) 

5.  Scientific  and  Professional 
Society  Membership  (43°o) 

13.  Current  Orqanizational 
Status  (38°0) 

6. *  Judged  Work  Output  (35%) 

8.*  Supervisory  Ratings  on 

Overall  Performance  (35°o) 

15.  Peer  Rankinqs  on 
Productivity  (35°o) 

l.4  Productivity  in  Written 
Work  (32°o) 

7.  Creativity  Ratinq  by  Laboratory 
Chiefs  (29°0) 


17.  Supervisory  Ratinq  of  Creativity  (29%) 

16.*  Supervisory  Rating  of  Drive- 
Resourcefulness  (25%) 

4.*  Oriqinality  of  Written 
Work  (20%) 

10.  Visibility  (20%) 

11.  Recognition  for  Orqanizational 
Contributions  (17%) 

2.  Recent  Publications  (14%) 

14.  Contract  Monitoring  Load  (11%) 

12. *  Status  Seeking,  "Organizational 

Man"  Tendencies  (08%) 

3.  Quality  (without  Originality)  of 
Research  Reports  (02%) 


If  all  of  the  30  empirically-keyed  scores  on  the  Bioqraphical  Information  Blank  are  omitted 
from  consideration,  the  criteria  rearrange  themselves  only  slightly  in  terms  of  their  percentage  of 
validity  coefficients  for  the  remaining  100  test  scores.  In  fact,  the  rank-order  correlation  between 
the  above  list  and  the  rearranged  list  was  .95. 

Among  the  less  predictable  criteria  are  Recognition  for  Organizational  Contributions,  Contract 
Monitorinq  Load,  and  Status-Seeking  Tendencies,  none  of  which  was  the  type  of  criterion  we  had 
most  directly  in  mind  as  we  developed  and  selected  psychological  test  scores  designed  to  predict 
creativity  and  productivity  as  scientists. 

It  is  admitted  that  the  distributions  found  for  several  of  the  initial  criterion  scores  were 
unusual  in  shape.  One  set  of  criterion  scores  warrants  special  comment  on  this  point.  The 
Recent  Publications  criterion,  which  measured  the  number  of  research  reports  in  a  two-year 
period,  was  an  unusual  and  unexpected  factor  found  in  the  criterion  analysis,  since  it  was 
separate  from  the  lifetime  total  of  articles  and  research  reports  found  in  the  first  criterion  score 
on  Productivity  ir.  Written  Work.  Apparently  the  Recent  Publications  criterion  arose  out  of  un¬ 
known  complexities  in  the  ebb-and-flow  periods  of  the  productivity  of  scientists.  Consequently 
many  scientists,  some  of  whom  had  a  high  written  productivity  index  over  their  life  span  to  that 
date,  were  found  to  have  a  score  of  zero  on  the  Recent  Publications  criterion.  This  criterion  was 
also  much  less  predictable  than  the  first  criterion  on  Productivity  in  Written  Work. 
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These  zero  scores  on  the  Recent  Publicotions  criterion  also  led  to  complications  on  two 
rating  criteria  of  these  two-year  research  reports,  namely,  Quality  (without  Originality)  of 
Research  Reports  and  Originality  of  Written  Work.  When  a  scientist  had  no  reports  for  the  two- 
year  period,  no  ratings  were  obtained  on  their  research  reports;  so  an  estimated  score  at  the 
middle  of  the  rating  scale  was  used  for  such  scientists.  That  is,  they  were  assigned  a  middle 
rating  on  the  quality  and  also  on  the  originality  of  their  research  reports.  These  complications  of 
using  many  estimated  scores  at  the  middle  of  the  rating  scale  probably  at  least  partly  account  for 
the  low  predictability  of  these  two  criteria  on  the  ratings  of  research  reports.  Since  the  Originality 
of  Written  Work  criterion  also  included  in  its  composite  the  patent  rate  and  the  number  of  officially 
accepted  suggestions  of  a  scientist,  it  was  more  predictable  than  the  criterion  of  Quality  (without 
Originality)  of  Research  Reports. 

Validity  of  the  Psychological  Tests.  The  types  of  test  scores  arrange  themselves  as  follows 
in  terms  of  the  percentage  of  significant  validities  (at  the  .05  level  of  significance)  for  all  their 
scores  against  all  of  the  criteria.  No  attempt  has  oeen  made  in  this  list  to  allow  for  possible 
restriction-of-range  effects  in  some  of  these  types  of  test. 


Per  Cent  of 

Type  of  Test  Scores  Volid 


Biographical  Information  Blank  (BIB)  (with  empirically  keyed  scores)  47 

Biographical  Information  Blank  (BIB)  (a  priori  keyed  scores,  only)  34 

Self  Ratings  (SR)  33 

Grade-Point  Average  24 

Minimum  Satisfactory  Level  (MSL)  22 

Profile  Matching9  20 

Motivational  Analysis  Test  (MAT)  8 

Personality  Research  Inventory  (PRI)  8 

Creative  Process  Check  List  6 

Aptitude  Tests  4 


The  Biographical  Information  Blank  was  clearly  the  best  all-round  single  instrument  for 
predicting  the  multiple  criteria  of  the  contributions  of  scientists.  It  was  decidedly  best  consider¬ 
ing  empirical  keys  (built  on  this  sample),  and  still  clearly  so,  using  only  a  priori  keys,  which 
were  based  partly  on  the  results  from  previous  studies  using  similar  items.  Extremely  high 
validities  on  the  main  sample  were  obtained  for  the  special  empirically-keyed  scores  as  expected 
for  these  spurious  correlations.  However,  no  cross  validation  was  attempted  since  it  was  decided 
that  the  entire  main  sample  of  107  scientists  should  be  used  as  a  basis  for  building  the  empirical 
keys.10  Further  evidence  of  the  merit  of  this  type  of  inventory  is  found  in  the  results  for  the  a 
prion  keyed  scores,  each  of  v/hich  was  obtained  from  only  15  or  20  selected  items,  with  no  items 
used  in  more  than  one  key.  An  average  of  nearly  6  out  of  17  of  the  validities  for  these  a  priori 
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in  Vernon  Corter's  unfinished  maters's  thesis  at  the  University  of  Utoh,  opproximotely  20%  of  the  volidity  coeffi¬ 
cients  lor  his  lfi  prolile-matchinq  scores  were  siqmlicont  oqainst  the  17  enteno  used  in  the  present  sludy.  On  the  bosis 
of  his  results,  four  profilo-motchinq  scores  were  selected  for  use  in  lhis  volidotion  study.  I'orty  per  cent  of  the  validities 
proved  siqmficanl,  a  smoll  percentaqe  of  which  were  spurious. 


Using  the  some  type  of  inventory  on  larger  somples  of  NASA  scientists,  we  hove  recently  lound  cross  validities 


in  the  .SO's  and  high  .40's  startinq  lrom  ahout  the  some  order  of  mogmtude  of  volidity  coefficients  os  found  lr,  the  present 
study  on  the  "mitiol  keyed"  somple. 
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keys  were  significant,  which  suggests  that  we  are  making  progress  in  learning  which  responses  to 
key  as  a  result  of  our  studies  of  scientists  and  their  biographical  characteristics. 

The  second  best  predictive  instrument  in  terms  of  the  number  and  magnitude  of  the  validity 
coefficients  was  the  Self-Ratings  on  17  scales.  In  fact,  the  12  best  rating  scales  had  an  average 
of  about  IV2  of  the  17  criteria  for  which  they  had  significant  validity  coefficients. 

The  Grade-Point  average  was  valid  for  only  4  of  the  17  criteria  and  just  barely  valid  for  3  of 
these  4. 

The  fourth  best  predictive  device  was  the  Minimum  Satisfactory  Level  scales  which  yielded 
12  scores.  The  8  best  scores  were  valid  for  an  average  of  approximately  5  out  of  the  17  criteria, 
and  an  MSL  score  was  the  most  valid  or  one  of  the  most  valid  scores  for  9  of  the  17  criteria.  This 
device  may  well  be  considered  the  third  rather  than  the  fourth  best,  since  these  12  scores  were 
very  rapidly  obtained  and  since  at  least  one  of  them  had  a  higher  validity  than  the  Grade-Point 
Average  on  every  criterion  for  which  the  Grade-Point  Average  was  valid. 

Except  for  occasional  scores  derived  from  each  test  the  other  predictive  tests  including  the 
Personality  Research  Inventory,  the  Motivation  Analysis  Test,  the  Creative  Process  Check  List, 
and  the  Aptitude  tests,  had  low  percentages  of  significant  validity  coefficients. 

The  low  percentage  of  validities  on  certain  tests,  especially  on  the  aptitude  tests,  was 
puzzling.  The  main  scores  for  Revision  II  and  the  Word  Association  test  had  each  been  valid  for 
18  and  16,  respectively,  out  of  the  27  situational  test  criteria  in  the  communication  abilities 
project  (C.W.  Taylor  et  a/.,  1958)  and  the  Visual  Imagery  test  scores  had  consistently  yielded 
sizable  validity  coefficients  in  on  electronics  course. 1 1  The  remaining  high-level  aptitude  tests 
were  selected  to  measure  presumably  important  characteristics  in  scientists  such  as  adaptive 
flexibility,  originality,  broadly-diffused  attention,  conceptual  foresight,  and  sensitivity  to 
problems. 

A  first  possibility  as  to  why  these  high-level  aptitude  test  scores  had  generally  low  validi¬ 
ties  was  restriction  of  range,  either  through  high  selection  and  homogeneity  of  the  sample  on  these 
characteristics  or  through  shorter  time  limits  on  the  tests.  Each  of  these  effects  which  narrow  the 
range  of  scores  may  have  occurred  in  some,  but  certainly  not  in  all,  instances.  For  example,  in 
the  communication  abilities  study,  the  ratio  score  of  words  per  idea  on  the  Revision  II  test  had  a 
mean  of  21.6  and  a  standard  deviation  of  4.1,  whereas  on  the  present  sample  these  statistics  were 
15.2  and  2.3,  respectively;  in  this  compactness  of  expression  score  the  group  of  scientists  per¬ 
formed  better  and  more  homogeneously  than  did  the  sample  in  the  communication  abilities  project. 
The  number  of  words  score  in  the  Word  Association  test,  however,  had  essentially  the  same  mean 
in  both  samples  and  a  somewhat  larger  standard  deviation  on  the  sample  of  scientists,  which  re¬ 
moves  the  possibility  of  the  restriction  of  range  explanation  forthis  test.  The  mean  score  on  the 
Visual  Imagery  test  was  12.92  for  the  scientists  which  was  from  2  to  5  points  higher  than  for 
comparison  groups  of  electronics  students  (no  standard  deviations  were  available  for  the  latter 
group).  Thus,  restriction  of  range  was  a  factor  in  the  Revision  II  and  the  Visual  Imagery  test 
scores  in  the  present  study. 

The  available  evidence  also  suggests  that  the  scores  on  the  Pertinent  Questions  and 
Apparatus  tests  were  restricted  in  range,  due  to  the  use  of  shorter  timelimits  for  the  scientists 
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Unpublished  World  War  II  Army  reports  by  L.layd  Lofquist,  Camp  Haod,  Texas,  1943-1944. 
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than  on  other  groups  tested  previously  by  Guilford.  A  half  length  (in  both  time  and  items)  form  of 
the  Pertinent  Questions  test  was  used,  while  a  shorter  timelimit  of  7  instead  of  10  minutes  was 
used  on  the  full  lenqth  Apparatus  Test.  However,  there  was  no  evidence  of  restriction  of  range 
on  the  particular  scores  on  the  Match  Problems  and  Consequences  tests  for  which  comparative 
data  were  available.  Nonetheless,  the  scientists  scored  higher  and  showed  more  variability  on 
the  Match  Problems  test  in  5  minutes  than  did  Guilford's  sample  of  air  cadets  and  student  officers 
in  12  minutes  (Guilford  ef  aL,  1952).  For  various  reasons  it  is  difficult  to  conclude  whether  there 
was  much  restriction  of  range  on  the  undergraduate  grude-point  averages,  where  the  scientists 
averaged  slightly  below  a  "B"  with  a  standard  deviation  of  half  a  qrade  point.  Of  course,  it  can 
always  be  argued  that  only  those  who  successfully  completed  college  are  on  the  job  in  scientific 
laboratories  (except  for  a  few  rare  cases).  The  reader  should  be  aware  that  in  estimating  cor¬ 
rections  to  overcome  restriction  of  range  effects,  in  most  cases  it  would  be  expected  that  validities 
which  were  essentially  zero  would  be  estimated  to  remain  zero  rather  than  to  be  significant 
validities  in  a  sample  not  showing  restriction  of  range.  In  other  words,  many  of  the  zero  validities 
may  be  truly  zero  validities  regardless  of  any  presence  or  adsence  of  restriction  of  range. 

A  second  partial  explanation  emerges  from  further  comparison  with  the  results  in  the  com¬ 
munication  abilities  study.  Both  the  Revision  II  and  the  Word  Association  tests  may  have  worked 
well  in  that  study  because  they  were  validated  against  well  controlled,  well  observed,  and  sys¬ 
tematically  scored  situational  test  criteria.  In  the  present  study  the  criterion  scores  were  generally 
obtained  from  more  naturalistic  and  less  well  standardized  observations  of  performance.  Perhaps 
aptitude  tests  predict  more  highly  structured  and  less  complicated  situational  test  scores  better 
than  they  predict  overall  ratings  and  other  types  of  on-the-job  criteria. 

The  other  evidenceon  the  above  matters  that  should  be  mentioned  is  that  the  communication 
scores  of  reading,  writing,  speaking,  and  listening  on  the  Minimum  Satisfactory  Level  scales 
were  also  used  in  the  communication  abilities  study.  The  means  and  standard  deviations  were 
essentially  comparable  across  both  studies.  About  half  of  their  validities  were  significant  in  the 
communication  abilities  study  whereas  slightly  less  than  one-fifth  of  the  validities  were  significant 
in  the  present  study,  which  generally  had  less  communication  features  in  its  multiple  criteria. 

It  is  admittedly  difficult  to  draw  firm  conclusions  about  these  proposed  explanations  for  the 
numerous  low  and  zero  validities,  when  so  many  variables  are  uncontrolled  in  the  comparisons 
available. 

The  results  on  the  Creative  Process  Check  List  were  somewhat  disappointing,  considering 
that  it  was  constructed  and  scored  on  the  basis  of  the  many  interestiig  introspective  writings  of 
famous  persons  about  their  creative  processes.  The  sta* es-of-attention  scorehad  no  significant 
validities  and  the  states-of-feeling  score  had  only  two  significant  validities,  and  these  were  quite 
unexpectedly  against  the  Visibility  criterion  and  the  Recognition  for  Organization  Contributions 
criterion,  neither  of  which  was  a  particularly  creative  type  of  criterion.  However,  the  states-of- 
feeling  score  was  clearly  the  most  valid  single  score  for  the  Recognition  criterion. 

CHARACTERISTICS  RELATED  TO  EACH  CRITERION 

The  results  will  next  be  presented  by  listing  the  characteristics  significantly  related  to  each 
criterion  and  to  each  control  score  in  sequence  from  the  hiqhest  to  the  lowest  related  character¬ 
istic.  In  this  description,  none  of  the  empirical-keyed  biographical  scores  developed  on  this 
sample  was  included  since  all  such  scores  correlated  spuriously  high  with  the  particular  criterion 
for  which  each  one  was  developed.  Biographical  scores  of  the  a  priori  type  are  included.  However, 
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the  profile  scores  are  not  included,  since  they  are  partly  situational  scores  and  would  not  be 
available  to  prospective  employees  as  predictor  scores.  The  instrument  from  which  each  score 
was  derived  will  generally  be  designated  by  an  abbreviation  in  parentheses  after  the  score.  The 
abbreviations  are  SR  for  Self  Ratings,  PRI  for  Personality  Research  Inventory,  MSL  for  the  aspi- 
rational  self  report  on  Minimum  Satisfactory  Level,  MAT  for  the  Motivation  Analysis  Test,  and 
BIB  for  an  a  priori  score  on  the  Biographical  Information  Blank. 

CRITERION  1.  PRODUCTIVITY  IN  WRITTEN  WORK 


Vor 

Nr 

P  rcdictor 

r 

Vor 

Nr 

P  redictor 

r 

141 

BIB 

Likinq  to  think 

.43 

89 

MSL 

Level  of  original  work 

.28 

8G 

MSL 

Quantity  a!  reports 

A  1 
•  t  i 

138 

BIB 

High  self-sufficiency 

.28 

1  35 

BIB 

Professional  self-canf idence 

.41 

34 

SR 

Discrimination  af  value 

.27 

67 

MSL 

Theoretical  contributions 

.38 

104 

GPA 

Grade-point  average 

.27 

37 

SR 

Creativity 

.34 

81 

MSL 

Writing  skills 

.26 

139 

BIB 

Inner  directedness 

.34 

145 

BIB 

Madal  response 

-.26 

142 

BIB 

Intellectual  thoroughness 

■  .32 

143 

BIB 

Social  desirability 

-.25 

140 

BIB 

Dedication  ta  wark 

.31 

47 

PRI 

Self-suff  iciency 

.23 

21 

SR 

Priv  e 

.30 

79 

MSL 

Speaking  skills 

.23 

82 

MSL 

Quantity  af  work  output 

.30 

26 

SR 

Desire  for  facts 

.19 

28 

SR 

Desire  for  discovery 

.29 

45 

SR 

Altruism 

-.19 

33 

SR 

Independence 

.28 

48 

SR 

Greg  aria  us  ness 

.19 

This  was  the  main  criterion  that  showed  restriction  of  range,  so  its  validities  would  generally 
be  higher  for  the  population  of  scientists  at  this  center  than  was  found  for  this  sample  of  the 
scientists.  There  was  a  sizable  number  of  characteristics  related  to  this  written  productivity 
factor,  most  of  them  being  very  meaningful  in  their  relationship.  It  should  be  recalled  from  the 
criterion  study  that  scientists  with  high  scores  on  this  criterion  were  generally  rated  below 
average  on  cooperation  by  their  supervisors. 

CRITERION  2.  RECENT  PUBLICATIONS 


Nr 

Predictor 

r 

43 

PRI 

Compulsiveness 

.28 

52 

PRI 

Belief  in  rights  of  individuals 

• 

ro 

42 

PRI 

Toleronce  of  ambiguity 

-.24 

41 

PRI 

T oleronce  of  frustration 

-.19 

69 

Revision:  Totol  ideas  retaired 

-.19 

135 

BIB 

Professional  self-confidence 

.19 

This  criterion  factor  seemed  to  be  a  separate,  short-range  measure  of  publication  productivity 
for  a  two-year  period.  It  did  not  particularly  relate  to  overall,  written  productivity  and  so  was  an 
unusual  and  unexpected  factor.  The  few  characteristics  related  to  it,  however,  shed  some  light 
on  its  nature.  Apparently,  persons  with  these  characteristics  felt  compelled  to  write  their  reports 
or  they  would  have  found  themselves  frustrated  in  a  more  complicated,  ambiguous  situation  which 
they  could  not  readily  tolerate. 


21 


CRITERION  3.  QUALITY  (WITHOUT  ORIGINALITY)  OF  RESEARCH  REPORTS 


Nr 

P  rcdictor 

r 

55 

PRI  Status  aspiration 

-.21 

95 

MAT  Self -sentiment 

.20 

143 

BIB  Sociol  desirobility 

.20 

The  positive  validity  for  the  social  desirability  score  was  atypical  for  this  study,  since 
every  other  time  that  it  is  valid  for  a  criterion,  its  validity  is  negative.  This  criterion  was  the 
least  predictable  criterion  in  the  study,  with  only  three  out  of  96  scores  correlating  with  it  sig¬ 
nificantly  (at  the  .05  level).  As  stated  earlier,  one  reason  why  this  criterion  was  not  very 
predictable  was  that  a  sizable  number  of  scientists  during  the  two-year  period  had  zero  publica 
tions  to  be  judqed  on  quality.  Consequently,  they  were  given  an  estimated  quality  score  at  the 
mean,  which  increased  the  difficulty  of  predicting  the  scores  on  this  criterion. 


CRITERION  4.  ORIGINALITY  OF  WRITTEN  WORK 


Nr 

Pred  ictor 

r 

44 

PRI 

Impulsiveness 

-.25 

139 

BIB 

Inner  directedness 

.23 

24 

SR 

Coqnit  ion 

.22 

37 

SR 

Creoti  vit  y 

.22 

Nr 

P  red ictor 

r 

61 

PRI  Progressive  (vs.  conservative) 

.21 

86 

MSL  Quantity  cf  reports 

.20 

97 

MAT  Sadism 

.19 

The  score  on  aspiration  of  level  of  original  work  was  slightly  below  the  level  of  significance. 
Even  though  non-publi^h^rs  during  the  two-year  period  were  given  estimated  scores  at  the  mean  on 
one  of  the  three  scores  constituting  this  criterion,  enough  correlates  with  meaningful  relations 
emerged  to  make  this  one  of  the  most  interesting  criteria  in  this  study.  These  valid  relations  tend 
to  be  in  expected  directions.  The  non-impulsiveness  finding  does  not  argue  that  originality  in 
paper  work  is  related  to  the  spontaneous  type  of  creativity,  currently  championed  by  some  specu¬ 
lators  in  creativity. 


CRITERION  5.  SOCIETY  MEMBERSHIP 


Nr 

Predictor 

r 

Nr 

Predictor 

r 

37 

MSL 

Theoretical  contributions 

.41 

82 

MSL 

Quantity  of  work  output 

.27 

87 

SR 

Creativity 

.39 

139 

BIB 

Inner  directedness 

.27 

81 

MSL 

Writing  skills 

.37 

91 

Apparatus  test:  total 

.26 

36 

SR 

Intuition 

.36 

31 

SR 

Flexibility 

.24 

89 

MSL 

Original  work 

.35 

33 

SR 

I nde  pendence 

.24 

145 

RIB 

Modal  response 

-.34 

35 

SR 

Cooperation 

.24 

29 

SR 

Information  ability 

.33 

53 

PRI 

Belief  in  rights  of  groups 

.2  3 

86 

MSL 

Quantity  of  reports 

.33 

141 

Bro 

Liking  to  think 

.22 

28 

SR 

Desire  far  discovery 

.33 

142 

BIB 

Fntejlectuol  thoroughness 

.21 

27 

SR 

Desire  for  principles 

.31 

21 

SR 

D”ive 

.20 

24 

SR 

Cognit  ion 

.30 

60 

PRI 

Spiritual  (vs.  natural) 

.21 

140 

BIB 

Dedication  t  o  work 

.29 

80 

MSL 

Listening  skills 

.20 

144 

BIB 

Self-reported  academic  level 

.2  8 

22 

SR 

Math  ability 

.19 

32 

SR 

Persistence 

.27 

25 

SR 

Integrity 

.20 

34 

SR 

Discrimination  of  value 

.27 

135 

BIB 

Professional  self-confidence 

.19 

22 


This  criterion,  quite  unexpectedly,  was  one  of  the  two  most  predictable  criteria  in  this 
study,  with  the  most  valid  characteristics  pertaining  to  creative,  theoretical  written  work.  The 
relationships  are  possibly  underestimates  because  of  some  restriction  of  range  found  earlier  in 
these  criterion  scores.  In  this  sample  of  scientists,  contrary  to  earlier  suggestions,  the  valid 
predictive  information  indicates  that  the  persons  with  the  more  creative  characteristics  are 
"belongers"  rather  than  "not-belongers"  to  professional  societies.  They  are  also  seekers  of  new 
principles;  they  desire  to  be  efficient  listeners;  and  they  have  drive,  persistence,  and  dedication 
to  their  work. 


CRITERION  6.  ACTUAL  WORK  OUTPUT  AS  JUDGED  BY  PEERS,  SUPERVISORS,  AND  CHIEFS 


Nr 

Predi ctor 

r 

Nr 

P  redi ctor 

r 

33 

SR 

Independence 

.46 

94 

MAT 

Assertion 

-.27 

34 

SR 

Discrimination  of  value 

.38 

36 

SR 

Intui  tion 

.25 

31 

SR 

Flexibility 

.35 

83 

MSL 

Being  well  liked 

.25 

24 

SR 

Cognition 

.34 

29 

SR 

Information  ability 

.25 

135 

BIB 

Professional  self-confidence 

.32 

I  38 

BIB 

Self-sufficiency 

.23 

37 

SR 

Creativi  ty 

.31 

1  36 

BIB 

Emotional  restraint 

.23 

27 

SR 

Desire  for  principles 

.28 

58 

PRI 

Masculine  vigor 

-.20 

21 

SR 

Drive 

.27 

89 

MSL 

Original  work 

.20 

23 

SR 

Resourcefulness 

.27 

54 

PRI 

Social  confidence 

.19 

28 

SR 

Desire  for  discovery 

.27 

139 

BIB 

Inner  directedness 

.19 

Persons  scoring  high  on  this  criterion  are  well  organized  and  motivated  toward  utilizing  their 
own  energies  and  efforts  efficiently  to  be  productive  in  their  work. 


CRITERION  7.  CREATIVITY  RATING  BY  LABORATORY  CHIEFS 


Nr 

Predictor 

r 

Nr 

Prod i ctor 

r 

141 

RIB 

Liking  to  think 

.30 

37 

SR 

Creativity 

.21 

80 

MSL 

Listening  skills 

.25 

104 

GP  A 

Grade-point  average 

.21 

1  38 

BIB 

Self-sufficiency 

.25 

78 

MSL 

Reading  skills 

.20 

101 

MAT 

Narcism  (comfort) 

.24 

140 

BIB 

Dedication  to  work 

.20 

136 

BIB 

Emotional  restraint 

.23 

24 

SR 

Cognit  ion 

.19 

139 

BIB 

Inner  directedness 

.23 

47 

PRI 

Self-sufficiency 

.19 

145 

BIB 

Modol  response 

-.23 

53 

PRI 

Belief  in  rights  of  groups 

.19 

21 

SR 

Drive 

.22 

94 

MAT 

Assertion 

-.19 

Several  scores  were  related  to  this  creativity  rating  by  the  higher  level  supervisor.  How 
ever,  only  a  few  of  these,  such  as  liking  to  think,  self-sufficiency,  drive,  and  a  self-report  on 
creativity  have  usually  been  reported  to  be  associated  with  creativity. 
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CRITERION  8.  QUALITY  OF  OVERALL  PERFORMANCE  AS  RATED  BY  SUPERVISORS 


Nr 

Predictor 

r 

Nr 

Predictor 

r 

21 

SR 

Drive 

.36 

2  3 

SR 

Resourcefulness 

.24 

34 

SR 

Discrimination  of  value 

.34 

27 

SR 

Desire  for  principles 

.24 

137 

BIB 

Low  sociability 

.34 

33 

SR 

Indepe  nde  nee 

.24 

24 

SR 

Cognition 

.33 

85 

MSL 

Being  well  known 

•  23 

83 

MSL 

Being  well  liked 

.30 

31 

SR 

Flexibility 

.22 

87 

MSL 

Theoretical  contributions 

.30 

1  42 

BIB 

Intellectual  thoroughness 

•  22 

37 

SR 

Creativity 

.29 

143 

BIB 

Social  desirability 

-.21 

86 

MSL 

Quontity  of  reports 

.27 

1  44 

BIB 

SeH-reported  academic  level 

.21 

139 

BIB 

Inner  directedness 

.27 

58 

PRI 

Masculine  vigor 

-.20 

49 

PRI 

Aggressiveness 

-.26 

22 

SR 

Moth  ability 

.19 

135 

BIB 

Profcssionol  self-confidence 

.26 

50 

PRI 

Attitude  to  work 

-.19 

145 

BIB 

Modol  response 

-.26 

84 

MSL 

Administrative  advancements 

.19 

36 

SR 

Intuition 

.25 

Many  characteristics  were  related  to  this  criterion  on  overall  quality  of  performance,  though 
no  relationship  was  very  strong.  The  direction  and  nature  of  the  drives,  aspirations,  and  attitudes 
of  those  high  on  this  criterion  are  illuminating. 


CRITERION  9.  LIKABLENESS  AS  A  MEMBER  OF  THE  RESEARCH  TEAM 


Nr 

P  redictor 

r 

Nr 

P  redictor 

r 

81 

MSL 

Writing  skills 

.46 

142 

BIB 

Intellectuol  thoroughness 

.27 

34 

SR 

Discrimination  of  value 

•  41 

28 

SR 

Desire  for  discovery 

.26 

37 

SR 

Creativity 

.34 

136 

BIB 

Emotionoi  restroint 

.26 

138 

BIB 

Self-suff  iciency 

.34 

141 

BIB 

Lik  ing  to  think 

.25 

31 

SR 

Flexibility 

.33 

85 

MSL 

Bei^g  well  known 

.24 

145 

BIB 

Modal  response 

-.33 

86 

MSL 

Quantity  of  reports 

.24 

24 

SR 

Cognition 

.31 

91 

Apparotus  test:  totol 

.24 

36 

SR 

Intuition 

.31 

13  5 

BIB 

Professional  self-confidence 

.23 

139 

BIB 

Inner  directedness 

.31 

100 

MAT 

Parents-home 

-.22 

27 

SR 

Desire  for  principles 

•  30 

42 

PRI 

Toleronce  of  ambiguity 

.21 

101 

MAT 

Norcism  (comfort) 

.30 

64 

Match  problems:  Nr  correct 

.2  0 

78 

MSL 

Reading  skills 

.30 

95 

MAT 

Self-sentiment 

.20 

59 

PRI 

Artistic  (vs.  practical) 

.28 

22 

SR 

Math  ability 

.19 

74 

Visuol  imogery:  %  marked  correct 

.28 

48 

PRI 

Gregariousness 

-.20 

79 

MSL 

Speaking  skills 

.27 

87 

MSL 

Theoreticol  contributions 

.19 

This  criterion  was  one  of  the  two  most  predictable  criteria  in  this  study.  Persons  with  these 
characteristics  were  judged  to  be  valuable  members  of  the  research  team. 
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CRITERION  10.  VISIBILITY 


Nr 

Predictor 

r 

Nr 

Predictor 

r 

H5 

BIB  Professional  self-confidence 

.30 

27 

SR 

Desire  for  principles 

.20 

77 

Word  ossociotion-  Nr  of  sets 

.23 

34 

SR 

Discr iminotion  of  volue 

.2  0 

103 

Creotive  process  check  list' 

37 

SR 

Creotivity 

.2  0 

Stotes  of  feeling 

.23 

44 

PRI 

Impulsiveness 

-.20 

31 

SR  Flexibility 

.22 

The  above  characteristics  describe  the  scientists  who  were  quite  visible  and  well  known 
throughout  this  research  installation. 


CRITERION  11.  RECOGNITION  FOR  ORGANIZATIONAL  CONTRIBUTIONS 


Nr 

P  redictor 

r 

Nr 

Predictor 

r 

103 

Creotive  process  check  list: 

73 

Visuol  imogery: 

States  of  feeLing 

.37 

wrong  attempts  +  skips 

-.20 

138 

BIB  Self-suf  f  iciency 

.29 

34 

SR  Discrimination  of  value 

.19 

96 

MAT  Wife,  sweetheort 

.24 

36 

SR  Intuition 

.19 

58 

PRI  Masculine  vigor 

-.22 

82 

MSL  Quontity  of  work  output 

.19 

I  37 

BIB  Low  sociability 

.21 

13  5 

BIB  Professional  self-confidence 

.19 

The  most  clearly  valid  characteristic  for  this  criterion  was  the  "states  of  feeling"  score 
from  the  Creative  Process  Check  List.  It  is  possible  that  additional  scoring  methods  will 
uncover  further  potential  in  this  check  list. 


CRITERION  12.  STATUS-SEEKING,  "ORGANIZATIONAL-MAN"  TENDENCIES 


Nr 

Predictor 

r 

82 

MSL  Quontity  of  work  output 

-.23 

25 

SR  Integrity 

-.21 

99 

MAT  Super  ego 

.21 

47 

PRI  Self-sufficiency 

-.20 

95 

MAT  Self-sentiment 

.21 

139 

BIB  Inner  directedness 

.20 

This  criterion  is  a  very  interesting  one  because  of  its  valid  characteristics  as  well  as  the 
directions  of  the  relations  that  were  found  for  these  few  characteristics.  In  recalling  the  first 
study  of  the  contributions  of  scientists,  it  was  found  that  promotion  rate  was  the  main  component 
in  this  factor.  A  low  number  of  suggestions  officially  accepted  by  the  organization  and  a  low 
self-rating  on  desire  for  discovery  were  positively  related  to  promotion  rate.  These  relations  in 
both  the  criterion  and  validation  studies  raise  some  real  puzzles  about  the  type  of  person  who  may 
have  a  fast  promotion  rate  in  a  research  organization.  It  will  be  seen  later  that  some  of  these 
characteristics  are  quite  different  and,  in  the  cases  of  self-sufficiency,  desire  for  discovery,  and 
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ospirotion  in  quontity  of  work  output,  ore  exoctly  opposite  to  those  found  for  scientists  who  are 
the  best  oil-round  contributors.  These  results  suqqest  thot  some  stotus-seekinq  orqanizationol 
men,  hoving  these  volues  of  their  own  os  they  function  in  reseorch  orqonizotions,  moy  occasion¬ 
ally  be  working  oqainst  the  truly  importont  qools  of  the  orqonizotion. 


CRITERION  13.  CURRENT  ORGANIZATIONAL  STATUS 


Nr 

Predictor 

r 

Nr 

Predictor 

r 

81 

MSL 

Writinq  skills 

•  47 

29 

SR 

Information  obility 

.23 

36 

SR 

Intuition 

.41 

138 

BIB 

Self-suf  fi  ci  ency 

.23 

34 

SR 

Discrimination  of  value 

.39 

144 

BIB 

Self-reported  ocodemic  level 

.22 

145 

BIB 

Modol  response 

-.38 

86 

MSL 

Quontity  of  reports 

.21 

142 

RIB 

Intellectual  thnrouqhness 

.37 

27 

SR 

Desire  for  principles 

.21 

135 

BIB 

Professional  self-conf idence 

.36 

73 

Visual  imoqery: 

24 

SR 

Coqnition 

.35 

wronq  ottempts  +  skips 

-.20 

37 

SR 

Creotivity 

.33 

74 

Visuol  imoqery:  %  correct 

.20 

87 

MSL 

Theoreticol  contributions 

•  33 

89 

MSL 

Onqi  nol  vv  ork 

.20 

33 

SR 

Independence 

.30 

91 

Apporotus  test:  totol 

.20 

48 

PR. 

Greqariousness 

-.28 

96 

MAT 

Wife,  sweetheort 

.20 

139 

BIB 

Inner  directedness 

.27 

78 

MSL 

Reo  di  nq 

.20 

31 

SR 

Flexibility  4 

.25 

47 

PRI 

Self-sufficiency 

.19 

85 

MSL 

Beinq  well  known 

.25 

28 

SR 

Desire  for  discovery 

.19 

23 

SR 

Resourcefulness 

.23 

104 

Grode-poinl  overoqe 

.19 

This  wos  the  third  most  predictoble  criterion  in  the  validotion  study.  The  most  related 
chorocteristic  wos  ospirotion  in  writinq  skills  (MSL)  which  overlapped  22%  of  the  voriotion  in  the 
criterion  scores.  In  summary,  the  person  with  hiqh  stotus  knows  thot  he  needs  better  writinq 
skills,  rotes  himself  os  hovinq  good  judgment,  is  thorouqh,  self-confident,  independent,  self- 
sufficient,  ond  willing  to  "live  alone  with  his  work"  much  of  the  time. 


CRITERION  14.  CONTRACT  MONITORING  LOAD 


Nr 

Predi  ctor 

r 

98 

MAT 

Corear 

.33 

46 

PRI 

T  oik  at  i  ve  ness 

.23 

54 

PRI 

Social  conscience 

.22 

58 

PRI 

Masculine  viqor 

.21 

13  9 

RIB 

Inner  directedness 

.19 

145 

BIB 

Modal  response 

-.19 

In  generol,  the  few  valid  characteristics  were  meoninqful  for  this  criterion,  ond  their  validi¬ 
ties  may  be  slightly  underestimated  becouse  of  some  indicotion  of  restriction  of  ronqe  in  the 
criterion  scores.  The  predominant  findinq  wos  thot  the  career  score  (MAT)  wos  importont  to  those 
with  o  heovy  load  of  controct  monitorinq,  being  the  only  time  thot  this  career  score  wos  volid  in 
the  study.  Likewise  the  only  time  thot  the  tolkativeness  score  (PR1)  wos  volid  wos  for  this  cri¬ 
terion.  Apporently  those  who  had  accepted  or  cultivated  heavy  monitorinq  loods  felt  somewhot  of 
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a  social  responsibility  to  do  so,  and  were  more  talkative  and  "career-oriented"  than  other 
scientists. 


Criterion  15.  Peer  Ranking  on  Overaii  Productivity.  This  was  the  first  of  three  initial 
criterion  scores  retained  in  addition  to  the  14  criterion  factors  or  categories,  which  were  each 
formed  by  a  composite  of  initial  criterion  scores.  Since  these  peer  rankings  were  the  most 
heavily  weighted  part  of  criterion  6,  Actual  Work  Output,  the  results  highly  overlapped  those  for 
criterion  6.  In  fact,  these  two  criteria  differed  only  in  two  sets  of  four  characteristics  which 
were  at  the  bare  level  of  significance  in  validity  on  one  criterion  and  below  that  level  on  the 
other,  while  16  characteristics  were  found  to  be  in  common  across  them.  This  finding  may  be 
reassuring  to  those  who  have  doubts  about  using  factor  criterion  scores  in  place  of  initially 
obtained  criterion  scores,  such  as  peer  rankings.  The  four  characteristics  barely  valid  for  peer 
rankings,  but  slightly  below  that  lev^l  for  criterion  6,  were  mathematical  ability  (SR),  under¬ 
graduate  grade-point  average,  intellectual  thoroughness  (BIB),  and  a  high  self-reported  academic 
level  (BIB).  Contrarily,  the  four  characteristics  that  were  barely  valid  for  criterion  6  but  slightly 
below  the  significance  level  of  validity  for  this  criterion  were  emotional  restraint  (BIB),  a  low 
score  on  masculine  vigor  (PRI),  social  confidence  (PRI),  and  inner  directedness  (BIB). 


CRITERION  16.  SUPERVISORY  RATING  OF  DRIVE-RCSOURCETULNESS 


Nr 

P  r  ed i c  t  o  r 

r 

Nr 

Predictor 

r 

137 

BIB 

Low  sociability 

.35 

58 

PRI 

Masculine  viqar 

-.2  2 

21 

SR 

Drive 

.32 

139 

BIB 

Ipner  directedness 

.22 

33 

SR 

Independence 

.24 

27 

SR 

Desire  for  principles 

.21 

143 

BIB 

Social  desirability 

-.24 

54 

PRI 

Social  conscience 

.21 

49 

PRI 

Aqressi  veness 

-.23 

23 

SR 

Resaurcef  ulnes  s 

.20 

37 

SR 

Creativit  y 

.22 

34 

SR 

Discrimina tion  of  value 

o 

CM 

This 

initial  criterion  was 

retained  separ 

ately  beca 

use  of  its  strong  motivational  nature. 

The 

crucial  question  was  whether  motivation,  which  is  often  claimed  to  be  so  important  in  job  perform¬ 
ance,  was  a  predictable  criterion.  The  answer  was  affirmative  with  the  very  interesting  finding 
that  many  of  the  valid  characteristics  were  motivational  in  nature.  The  type  of  motivation  that  is 
effective  in  this  scientific  work  is  apparently  more  of  a  quiet,  independent,  task-focused  power, 
not  an  outwardly  socially  persuasive  or  socially  aggressive  power,  not  a  masculine,  physically 
vigorous  power.  It  is  very  interesting  that  both  the  criterion  ratings  on  drive  and  on  resourcefulness 
together  with  several  predictor  scores  that  were  highly  motivational  in  nature  all  clustered  together, 
though  their  interrelations  were  not  high,  nor  were  their  validities  very  high  aqainst  any  criterion 
in  this  project.  Although  the  aspiration  scores  generally  showed  good  results  for  other  criteria, 
all  of  them  were  missing  from  this  cluster  and  had  no  noticeable  validity  in  predicting  supervisory 
ratings  of  motivation. 

These  findinqs  clarify  somewhat  the  entire  motivational  problem.  First,  motivational  scores 
tend  to  cluster  somewhat  loosely  in  two  or  more  clusters.  Secondly,  motivation  is  either  a  minor 
contributor  (not  one  of  the  major  contributors)  to  effective  performance  on  the  job  or  else  motivation 
is  currently  being  measured  at  a  very  low  level  of  efficiency. 
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CKITEHION  17.  SUPFZHVISOHY  BATING  OP  CHEATIVITY 


Nr 

Predictor 

r 

Nr 

P  redictor 

r 

137 

BIB 

Low  sociability 

.32 

144 

BIB 

Sell-reported  ocademic  level 

.24 

87 

MSL 

Theoretical  contribu tions 

.31 

145 

BIB 

Modal  response 

-.24 

83 

MSL 

Beinq  we!'  liked 

.29 

34 

SB 

Discrim inotion  ol  volue 

.2  3 

49 

PHI 

Aggressiveness 

-.28 

37 

SB 

Creativity 

.23 

1  38 

BIB 

Sel  1-sullicienc  y 

.25 

135 

BIB 

Prolessionol  sell-conl idence 

.23 

86 

MSL 

Quontity  ol  reports 

.26 

142 

BIB 

Intellectual  thoroughness 

.2  2 

21 

SB 

Drive 

.25 

89 

MSL 

Originol  work 

.21 

1  39 

BIB 

Inner  directedness 

.24 

140 

BIB 

Dedicotion  to  work 

.1  9 

This  initial  criterion  was  sinqled  out  because  of  the  special  interest  in  creativity.  Several 
characteristics  were  related  to  this  criterion  rating  of  creativity,  some  of  which  have  emerged  in 
other  studies.  Some  of  the  lowest  validities  in  the  above  list  plus  the  appearance  of  other  charac¬ 
teristics  like  self-sufficiency,  drive,  and  low  sociability  give  some  reassurance  that  the  supervisory 
rating  is  based  somewhat  on  creativity.  However,  the  appearance  of  other  characteristics  suggest 
that  some  features  other  than  creativity  are  entering  into  this  criterion  rating,  a  conclusion  also 
supported  by  the  high  relation  between  this  creativity  rating  and  other  supervisory  ratings  in  the 
criterion  study. 

Control  Score  Id.  Age.  None  of  the  a  priori  biographical  scores  and  only  a  few  of  the  other 
scores  were  related  to  age.  The  older  scientists  made  fewer  correct  responses  on  Visual  Imagery 
(a  difficult  visualizing,  imaginative  task),  and  had  higher  aspirations  in  being  well  known  (MSL), 
had  lower  scores  both  on  impulsiveness  (PRI)  and  on  aggressiveness  (PRI),  had  higher  scores  on 
social  know-how  (PRI),  raised  more  questions  on  Pertinent  Questions,  rated  themselves  as  more 
resourceful  (SR),  and  retained  a  higher  percentage  of  the  given  ideas  in  the  Revision  II  test. 

Since  age  was  not  highly  related  to  any  of  these  scores  and  it  was  also  not  particularly  related  to 
any  of  the  criterion  scores,  the  conclusion  in  this  study  is  that  age  is  not  a  very  relevant  variable, 
the  effect  of  whicn  does  not  need  to  be  controlled  or  somehow  partialled  out.  In  other  words,  very 
few  relations  found  in  the  study  wjuld  be  materially  altered  if  the  effects  of  age  were  either  deliber¬ 
ately  controlled  or  statistically  partialled  out. 

Control  Score  19.  Total  Years  ol  Work  Experience.  Only  a  few  characteristics  were  related 
to  total  work  experience.  Those  with  more  work  experience  rated  themselves  as  more  resourceful, 
more  skillful  in  scientific  techniques,  more  discriminating  in  value,  more  cognitive,  and  more  in¬ 
dependent  (all  SR).  They  scored  below  average  in  liking  to  think  (BIB)  and  in  the  number  of  correct 
responses  on  the  Visual  Imagery  test,  and  above  average  in  the  "states  of  feeling"  score  in  the 
Creative  Process  Check  List,  in  independence  (SR),  and  in  aspiration  in  being  well  known  (MSL). 

As  in  the  case  of  the  control  variable  of  Age,  few  relations  in  this  study  would  be  noticeably 
changed  if  the  effects  of  total  years  of  work  experience  were  either  controlled  or  partialled  out.  If 
this  variable  is  viewed  alternately  as  a  predictor  variable  (which  is  true  whenever  total  experi¬ 
ence  is  requested  in  an  application  blank  or  interview),  it  was  readily  seen  in  the  earlier  criterion 
study  that  it  had  essentially  zero  validity  for  each  of  the  criterion  factors  except  for  a  slight 
negative  validity  with  promotion  rate.  Consequently  there  is  little  evidence  from  this  project  that 
total  years  of  work  experience  should  be  used  as  selection  or  placement  information. 
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Control  Score  20.  Total  Months  o  1  Experience  at  This  Research  Center.  Only  5  out  of  96 
scores  were  related  significantly  (at  the  5 °o  level)  to  this  control  variable;  so  these  findings 
generally  do  not  warrant  much  attention  nor  is  there  great  need  to  control  this  variable  in  this 
study.  Those  with  more  experience  in  this  organization  rated  themselves  as  more  resourceful 
(SR),  had  below  averoge  scores  both  on  liking  to  think  (BIB)  and  on  tolerance  of  ambiguity  or 
complexity  (PRI),  and  above  average  scores  on  percentage  correct  on  Visuol  Imagery  and  on  the 
number  of  different  sets  taken  on  the  Word  Association  test. 

Years  o i  Education.  The  other  variable  that  could  be  singled  out  for  mention  is  the  years  of 
education.  It  is  usually  considered  as  predictor  information  to  be  gathered  on  application  forms 
and  in  interviews.  It  was  found  in  this  study  to  be  unrelated  to  all  criterion  factors  except  criterion 
1  on  the  effectiveness  in  paper  work  and  criterion  5  on  scientific  and  professional  society  member¬ 
ship  for  which  it  is  often  a  prerequisite.  In  other  words,  years  of  education  was  not  a  volid  score 
for  12  of  the  14  criterion  factors  and  it  can  be  viewed  either  as  a  valid  predictor  of  only  the  other 
two  criterion  factors  or  as  a  variable  whose  effects  should  be  cancelled  out  in  these  two  —  ond 
only  in  these  two  criteria.  As  a  safety  precaution,  however,  it  might  be  added  that  slightly  stronger 
relationships  and  a  few  more  relationships  might  emerge  if  a  wider  range  of  years  of  education  were 
present  in  a  sample  studied.  The  same  argument  might  be  relevant  in  case  of  o  wider  ronge  of  oges 
in  the  sample. 

CHARACTERISTICS  LEADING  TO  MULTIPLE  CONTRIBUTIONS  IN  SCIENCE 

The  following  characteristics  were  most  valid  for  predicting  across  the  multiple  criterio  in 
this  validation  study.  The  number  of  the  17  criteria  for  which  eoch  predictor  score  wos  valid  is 
listed  below.  For  example,  o  high  score  on  creativity  would  predict  a  person  to  be  an  above- 
average  contributor  in  1  2  of  the  17  criterion  scores. 


Var 

Nr 

Predicto  r 

Nr 

Volid 

Var 

Nr 

Predi 

ictor 

Nr 

Vol  i  d 

37 

SR 

Creativity 

12 

29 

SR 

Informative  ability 

4 

1  39 

BIB 

Inner  directedness 

12 

47 

PRI 

Self-sufficiency 

4 

34 

SR 

Disci imination  of  value 

11 

81 

MSL 

Writing  skills 

4 

135 

BIB 

Professional  self-canf idence 

1  1 

82 

MSL 

Quantity  af  work  output 

4 

24 

SR 

Coqnition 

9 

83 

MSL 

Being  well  liked 

4 

27 

SR 

Desire-f  ar-princi  pies 

9 

104 

Grade-point  average 

4 

21 

SR 

Drive 

8 

1  37 

BIB 

Low  sociability 

4 

1  38 

BIB 

Self-sufficiency 

8 

140 

BIB 

Dedication  to  wark 

4 

145 

BIB 

Modal  response  (law  score) 

8 

1  41 

BIB 

Liking  to  think 

4 

31 

SR 

Flexibility 

7 

143 

BIB 

Social  desirability  (law  score) 

4 

33 

SR 

Independence 

7 

48 

PRI 

Gregariousness  (low  score) 

3 

36 

SR 

Intuition 

7 

49 

PRI 

Aggressiveness  (low  score) 

3 

86 

MSL 

Quantity  af  reports 

7 

54 

PRI 

Social  conscience 

3 

87 

MSL 

Theoretical  contributions 

7 

78 

MSL 

Reading  skills 

3 

89 

MSL 

Level  of  original  wark 

7 

85 

MSL 

Being  well  known 

3 

142 

BIB 

Intellectual  thoroughness 

7 

91 

Apparatus  test:  fatal 

3 

28 

SR 

Desire  far  discovery 

6 

94 

MAT  Assertion  (low  score) 

3 

23 

Resourcefulness 

5 

1  36 

BIB 

Emotional  restraint 

3 

58 

PRI 

Masculine  vigor  (low  score) 

5 

1  44 

BIB 

Self-reported  ocademic  level 

5 

22 

SR 

Math  ability 

4 

29 


In  the  study  plan,  it  was  decided  to  compare  the  validities  of  different  ways  of  measuring 
personality  and  motivation.  The  results,  in  general,  were  rather  clearcut  in  both  of  these  compari¬ 
sons.  In  measuring  personality  characteristics  the  biographical  inventory  with  either  of  its  scoring 
systems  appeared  to  be  better  than  the  self-ratings,  which  in  turn  were  clearly  better  than  the  scores 
from  the  usual  type  of  personality  inventory.  It  should  also  be  noted  that  the  differences  in  validity 
could  not  be  accounted  for  in  terms  of  differences  in  lengths  of  instruments. 

In  the  motivational  area,  the  self-report  on  the  minimum  satisfactory  level  of  achievement, 
even  though  very  short  in  time  and  length,  clearly  yielded  more  valid  scores  than  either  a  new  and 
complex  motivational  test  or  the  two  intellectual  measures  of  motivation  which  involved  the  raising 
of  questions  and  the  sensing  of  problems.  A  very  short  activity-level  score  across  four  biographical 
items  failed  to  yield  positive  results,  although  to  our  knowledge  no  serious  attempt  has  yet  been 
made  to  develop  a  set  of  biographical  items  designed  to  yield  one  or  more  motivational  scores. 

One  puzzling  result  was  the  relative  lack  of  validity  of  any  of  the  multiple  scores  from  the 
seven  high-level  aptitude  tests.  This  finding  occurred  after  we  had  found  many  positive  results 
for  high-level  aptitude  tests  against  criteriaof  communication  abilities  (C.W.  Taylor  et  al.,  1958). 

In  the  previous  study,  however,  the  criteria  were  scores  of  performance  in  standardized  situational 
tests,  whereas  no  such  criteria  were  used  in  the  current  study  even  though  over  50  criterion  scores 
were  collected.  The  criteria  used  were  ratings,  rankings,  and  other  forms  of  judgments  of  scien¬ 
tists,  scores  from  official  records,  and  quantity  as  well  as  quality  judgments  of  products  of  scien¬ 
tists.  A  study  is  needed  to  determine  whether  high-level  aptitude  scores  predict  scientific  per¬ 
formance  in  standardized  situational  tests  better  than  they  predicted  any  of  the  present  criteria. 

If  they  do,  there  would  still  be  the  challenging  question  of  the  relation  of  judged  performance  in 
standardized  situations  to  judgments  about  performance  on  the  job,  and  the  question  would  be 
particularly  difficult  if  this  relation  were  found  to  be  low. 

CHARACTERISTICS  LEADING  TO  CREATIVE  CONTRIBUTIONS 

Six  criteria  (1,  4,  6,  7,  8,  17)  were  singled  out  because  of  some  creative  feature  in  them. 

The  characteristics  listed  below  were  found  to  be  most  frequently  related  to  the  six  criteria. 


Vor 

Nr 

Predictor 

Nr 

V  al  i  d 

37 

SR 

Creati  vity 

6 

139 

BIB 

Inner  directedness 

6 

21 

SR 

Drive 

5 

24 

SR 

Coqnition 

5 

13  5 

BIB 

Professional  self-confidence 

4 

34 

SR 

Discrimination  of  volue 

4 

86 

MSL 

Quontity  of  reports 

4 

87 

MSL 

Theoretical  contributions 

4 

138 

RIB 

Hiqh  self-sufficiency 

4 

1  45 

BIB 

Modol  response  (low  score) 

4 

33 

SR 

Independence 

3 

89 

MSL 

Level  of  oriqinol  work 

3 

142 

RIB 

Intellectual  thoroughness 

3 

27 

SB 

Des  ire-f  or-principles 

3 

Vor 

Nr 

P  r edi  ctor 

Nr 

Valid 

83 

MSL 

Beinq  well  liked 

3 

140 

BIB 

Dedication  to  work. 

3 

23 

SR 

Resourcefulness 

2 

28 

SR 

Des  ire-f  or -discovery 

2 

31 

SR 

Elexibility 

2 

36 

SR 

Intuition 

2 

47 

PRI 

Self-sufficiency 

2 

49 

PRI 

Aggressiveness  (low  scare) 

2 

58 

PRI 

Masculine  viqor  (low  score) 

2 

94 

MAT 

Assertion  (low  score) 

2 

104 

Grad< 

?-point  overage 

2 

136 

BIB 

Emotional  restrolnt 

2 

137 

BIB 

Low  sociability 

2 

141 

BIB 

Likinq  to  think 

2 

143 

BIB 

Social  desirability  (low  score) 

2 

I  44 

BIB 

Self-reported  acodemic  level 

2 

30 


Many  of  these  findings  are  consistent  with  the  research  reported  on  scientists  at  the  Utah 
creativity  conference  series.  In  fact,  a  pattern  of  characteristics  c i  low  sociability,  liking  to 
think  with  preference  for  ideas  over  things  and  people,  self-sufficiency,  independence,  intuitive¬ 
ness,  flexibility,  resourcefulness,  femininity,  and  dedication  to  their  work  with  high  productivity 
has  emerged  as  characteristics  of  the  more  creative  scientists  in  several  different  studies 
(Knapp,  1956;  Barron  5c  Roe,  1958) 


SUMMARY  AND  CONCLUSIONS 

Studies  on  samples  of  Air  Force  scientists  accomplished  in  a  three-year  project  have  been 
presented  in  this  report. 

The  first  study  entailed  intensive  interviews  of  nearly  200  scientists  which  yielded  volumi¬ 
nous  interview  reports  containing  considerable  information  of  value  in  all  of  the  other  studies  in 
the  project. 

The  turnover  sample  of  scientists  was  compared  with  the  non-turnover  sample  and  those 
scientists  who  cooperated  fully  on  this  project  were  compared  with  those  who  for  one  reason  or 
another  were  unable  to  cooperate  fully.  In  both  comparisons  on  criteria  of  performance  on  the  job, 
the  two  pairs  of  groups  were  more  alike  than  different,  with  few  significant  differences.  The 
main  sample  of  scientists  were  more  effective  paper  workers  than  either  the  turnover  or  the  non¬ 
cooperating  scientists.  The  turnover  sample  was  somewhat  younger,  less  experienced,  and  not  as 
fully  established  in  this  organization  as  the  non-turnover  group  of  scientist.  The  group  who  did 
not  cooperate  fully  had  more  work  experience  at  this  research  center  and  had  a  lower  score  on 
professional  society  membership  than  the  fully  cooperative  group. 

In  the  criterion  study,  the  multiple  contributions  of  a  sample  of  166  physical  scientists,  52 
criterion  scores,  were  analyzed,  with  the  finding  that  these  contributions  were  largely  accounted 
for  in  terms  of  14  main  categories.  These  were  productivity  in  written  work,  recent  research 
reports,  quality  (without  originality)  of  research  reports,  originality  of  written  work,  professional 
society  membership,  judgment  of  actual  work  output,  creativity  ratings  by  higher  level  supervisors, 
overall  quality  ratings  by  immediate  supervisors,  likableness  as  a  member  of  the  research  team, 
visibility,  recognition  for  organizational  contributions,  status-seeking  tendencies,  current  organ¬ 
izational  status,  and  contract  monitoring  load.  Age  and  work  experiences  were  found  to  be 
generally  unrelated  to  the  above  categories  of  contributions.  Years  of  education  was  also  found 
to  be  unrelated  to  the  scientist's  contributions,  with  the  exception  that  it  was  somewhat  related 
positively  to  effectiveness  in  paper  work  and  to  professional  society  membership. 

The  validation  study  was  completed  on  107  of  the  above  scientists,  using  130  predictor 
scores  and  17  criterion  scores  (one  for  each  of  the  14  categories  plus  peer  rankings  on  overall 
productivity,  supervisory  ratings  on  drive-resourcefulness,  and  supervisory  ratings  on  creativity). 

For  every  criterion  there  were  several  predictors  with  low  to  moderate  validity.  If  an  inde¬ 
pendent  sample  of  scientists  were  available,  effectiveness  of  best  combinations  of  predictors 
could  be  tested.  The  most  predictable  criteria,  in  terms  of  number  of  valid  scores,  were  likable¬ 
ness  as  a  member  of  the  research  team,  professional  society  membership,  current  organizational 
status,  judged  work  output,  supervisory  ratings  on  overall  performance,  and  peer  rankings  on 
overall  productivity. 

The  psychological  scores  found  to  be  valid  for  each  separate  criterion  have  been  described. 
In  addition,  40  characteristics  were  listed,  each  of  which  was  related  to  several  of  the  17  criteria. 
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The  scores  that  were  valid  for  the  largest  number  of  these  criteria  were  those  for  creativity,  inner 
dircctedness,  discrimination  of  value,  professional  self-confidence,  cognition,  desire  for  principles, 
drive,  self-sufficiency,  flexibility,  independence,  intuition,  aspiration  in  quantity  of  research  re¬ 
ports,  aspirotion  in  theoretical  contributions,  aspiration  in  high  level  of  original  work,  and  intel¬ 
lectual  thoroughness. 

The  most  valid  psychological  tests  were  the  bioqraphical  inventory  (both  with  empirical  and 
with  a  priori  keyed  scores),  self-rotinqs,  and  a  motivational  device  designed  to  measure  minimum 
satisfactory  level  of  aspiration.  Undergraduate  grade-point  average  correlated  significantly  with 
only  four  of  the  17  criteria,  with  three  of  these  barely  at  the  level  of  significance. 

No  attempt  has  been  made  in  this  project  to  predict  the  important  factor  of  turnover  of  scien¬ 
tists.  The  evidence  suggests  that  the  turnover  criterion  is  relatively  separate  from  the  other 
criteria  in  the  study.  Further  analysis  of  the  avoilable  data  is  needed  to  determine  the  character¬ 
istics  that  predict  turnover  or  non-turnover  and  to  see  how  these  characteristics  are  reloted  to 
those  that  predict  the  more  successful  performances  on  the  job. 

This  exploratory  investigation  has  identified  a  wide  range  of  criteria  and  their  possible 
predictors.  Further  work  is  needed  to  determine: 

(a)  which  of  the  criteria  and  predictors  provide,  in  a  longitudinal  situation,  satisfactorily 
stable  and  reliable  measurement; 

(b)  how  to  combine  reliable  meosures  of  scientific  contributions  as  criteria  of  accomplish¬ 
ment; 

(c)  whether  test  combinations  and  empirical  keys  for  self-report  predictor  instruments  can 
be  developed  that  are  valid  across  several  scientific  disciplines. 
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APPENDIX  A 
Statistical  Tabulations 
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Consultant 

42 

-15 

11 

22 

-39 

-25 

13 

-12 

04 

-09 

-07 

19 

05 

02 

16 

12 

05 

13 

04 

62 

46 

Truatworthy 

20 

-10 

-12 

03 

-10 

05 

05 

14 

-05 

-10 

-03 

19 

16 

08 

04 

08 

-24 

-04 

-07 

25 

47. 

Lab  nomln. 

67 

03 

33 

06 

17 

-15 

-13 

15 

26 

05 

-02 

-04 

-07 

11 

01 

06 

16 

02 

07 

78 

48. 

Cntr  nomln. 

54 

li 

36 

-06 

14 

07 

-22 

27 

-19 

05 

0.3 

-12 

06 

-09 

12 

10 

-04 

06 

06 

69 

49 

19 

-18 

33 

50 

-16 

35 

13 

13 

26 

-17 

09 

18 

-15 

09 

-10 

09 

-05 

08 

-11 

82 

50. 

Educ.,  yrs 

35 

31 

-11 

-17 

12 

09 

17 

-06 

11 

04 

04 

16 

12 

-08 

11 

19 

12 

22 

06 

49 

51. 

Work  yrs. 

21 

-23 

38 

-53 

21 

33 

09 

15 

22 

-16 

04 

13 

-13 

11 

-13 

-05 

08 

04 

-04 

86 

52. 

Months,  Cntr 

15 

-26 

38 

-31 

-20 

19 

-02 

25 

03 

-10 

-11 

-10 

11 

13 

-14 

-03 

-10 

-12 

08 

58 

39 


TABLE  5.  Transformation  Matrix  in  the  Criterion  Study 


A 

B 

C 

D 

E 

F 

G 

H 

1 

J 

K 

L 

M 

N 

0 

1 

-12 

-11 

-02 

-08 

-09 

-21 

-14 

-34 

-21 

-23 

-08 

-03 

-10 

-07 

-12 

11 

33 

14 

20 

18 

17 

-11 

23 

-53 

11 

12 

-02 

-03 

-18 

03 

-15 

III 

-14 

-14 

17 

10 

-13 

20 

20 

-25 

22 

29 

16 

-12 

22 

09 

29 

IV 

00 

-12 

25 

14 

-14 

21 

21 

23 

23 

-13 

-14 

11 

-22 

07 

-33 

V 

-10 

-07 

12 

-29 

10 

-48 

19 

28 

05 

20 

20 

16 

26 

-20 

-27 

VI 

-26 

-11 

28 

03 

08 

-26 

30 

24 

-20 

09 

06 

-23 

-27 

-09 

42 

VII 

03 

20 

39 

24 

07 

04 

-19 

28 

-21 

-35 

-23 

-22 

17 

17 

20 

Vlll 

28 

21 

01 

-04 

-34 

-10 

-27 

00 

27 

30 

17 

-23 

-43 

17 

29 

IX 

34 

-17 

-28 

-09 

-18 

00 

31 

-17 

18 

-45 

-37 

-10 

35 

19 

33 

X 

44 

14 

-06 

-31 

10 

31 

09 

-18 

10 

04 

13 

-08 

18 

26 

-29 

XI 

-11 

28 

-31 

24 

-11 

-24 

41 

19 

-38 

-02 

08 

24 

-19 

-12 

17 

Xll 

07 

-15 

05 

-65 

-20 

28 

25 

02 

14 

-29 

20 

-17 

-34 

09 

16 

XIII 

-43 

49 

02 

01 

-18 

-20 

32 

19 

-32 

04 

-14 

-13 

26 

26 

-17 

XIV 

11 

31 

31 

-22 

-19 

-07 

19 

13 

-11 

-25 

-16 

22 

10 

-28 

25 

XV 

25 

-25 

-08 

21 

-28 

-28 

07 

11 

-20 

-20 

34 

-44 

23 

-47 

-04 

XVI 

31 

-05 

42 

-21 

-28 

-23 

-27 

-30 

-35 

37 

13 

46 

19 

04 

XVII 

-01 

-32 

-19 

-08 

-13 

-03 

-04 

-10 

-43 

-10 

-43 

-29 

-15 

-21 

-09 

XV11I 

02 

31 

-35 

-23 

49 

-25 

15 

-08 

-07 

04 

-19 

-25 

-11 

-01 

-07 

XIX 

12 

-30 

-03 

-02 

-47 

28 

14 

-05 

-11 

17 

-47 

-27 

03 

-58 

-14; 

Note.  —  Decim al  points  have  been  amitled. 
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Recognition  foi  orgonizotionoi  contributions 
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TABLE  7.  Inter  cor  relotions.  Means,  and  Standard  Deviations  of  Criterion  Scares 


Crit 


Nr 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16  17 

Mean 

SD 

1 

-- 

51.02 

7.05 

2 

08 

-- 

1.01 

1.94 

3 

09  * 

*05 

-• 

49.69 

8.26 

4 

22 

13 

01 

-- 

21.49 

3.88 

5 

22 

07 

07 

19 

-- 

11.69 

9.38 

6 

24 

17 

03 

37 

24 

-- 

49.78 

6.71 

7 

32 

14 

22 

26 

18 

32 

-- 

16.51 

3.34 

8 

26 

28 

-03 

18 

25 

53 

33 

-- 

69.75 

15.61 

9 

33 

06 

13 

19 

27 

42 

31 

41 

-- 

50.73 

6.11 

10 

16 

27 

04 

32 

13 

41 

21 

33 

42 

-- 

50.14 

7.24 

11 

20 

10 

02 

-05 

10 

28 

11 

23 

25 

4P 

-- 

49.18 

4.53 

12 

-05 

-01 

02 

03 

06 

00 

08 

11 

18 

-03 

-05 

-- 

11.56 

5.10 

13 

23 

11 

-08 

08 

30 

38 

13 

33 

55 

42 

35 

04 

-- 

49.37 

3.01 

14 

09 

39 

08 

12 

12 

22 

10 

12 

19 

25 

12 

05 

25 

-- 

1.85 

3.37 

15 

26 

14 

00 

34 

20 

94 

29 

49 

40 

39 

24 

00 

38 

14 

-- 

52.58 

15.71 

16 

16 

33 

-01 

27 

20 

52 

33 

83 

30 

25 

11 

04 

24 

16 

45 

-- 

8.25 

2.22 

17 

29 

13 

-12 

18 

19 

31 

33 

79 

33 

36 

19 

15 

30 

OO 

30 

55  -- 

3.69 

1.20 

Note.  —  Decim ol  points  hove  been  omitted  precedinq  correlation  coefficients. 
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TABLE  8.  Predictor  Scores  Used  in  the  Volidotion  Study 


Test  derived 

from  Nr  Description  of  score  _ 

Self  Reports 

21  Drive  —  So  If -rating  on  7*point  scole 

22  Math  obility  —  Self-rotinq  on  7-point  scale 

23  Resourcefulness  —  Self-ratinq  on  7-point  scale  "* 

24  Coqnition  —  Self-ratinq  on  7-point  scale 

25  lnteqrity  —  Self-ratinq  on  7-point  scale 

26  Desire-for-f  acts  —  Sel  f-rotinq  on  7-point  scale 

27  Desire-for-principles  —  Self-ratinq  on  7-point  scale 

28  Desire-for-discovery  —  Self-ratinq  on  7-point  scale 

29  Informative  ability  —  Self-ratinq  on  7-point  scoJe 

30  Skill  —  Self-rotinq  on  7-point  scale 

31  Flexibility  —  Self-ratinq  on  7-point  scole 

32  Persistence  —  Self-rotinq  on  7-point  scale 

33  Independence  —  Self-rotinq  on  7-point  scale 

34  Discrimination  of  value  —  Self-ratinq  on  7-paint  scale 

35  Cooperation  —  Self-ratinq  on  7  point  scale 

36  Intuition  —  Self-rotinq  on  7-point  scale 

37  Creativity  —  Self  rotinq  on  7-point  scole 

Personality  Research  Inventory  (PRI) 

38  Self-insiqht  —  Number  riqht 

39  Free-f loatinq  onxiety  —  Number  riqht 

40  Sel f-o cceptance  —  Number  riqht 

41  Tolerance  of  f rustration  —  Number  riqht 

42  Tolerance  of  ambiquity  or  complexity  —  Number  riqht 

43  Compulsiveness  -  Number  riqht 

44  Impulsiveness  —  Number  riqht 

45  Altruism  —  Number  riqht 

46  Tolkativeness  —  Number  riqht 

47  Self-5>uff iciency  —  Number  nqht 

48  Greqariou sness  —  Number  nqht 

49  Aqqressi veness  —  Numbe-  nqht 

50  Attitude  to  work  —  Number  riqht 

51  Foresiqht  —  Number  nqht 

52  Relief  in  nqht  of  individuals  —  Number  nqht 

53  Relief  in  riqhts  of  qroups  —  Number  riqht 

54  Sociol  conscience  —  Number  riqht 

55  Stotus  aspirat ion  —  Number  riqht 

56  Social  know-ho w  —  Number  riqht 

57  Social  stot  us  —  Number  riqht 

58  Masculine  viqor  — Number  riqht 

59  Artistic  vs.  procticol  —Number  riqht 

60  Spiritual  vs.  moterial  —  Number  riqht 

61  Proqressive  vs.  conservative  —  Number  riqht 

62  Likmq  to  think  —  Number  riqht 

63  PRI  o cquiescence  —  Number  of  yes  responses 

Motch  Problems 

64  Number  of  correct  responses  (a) 

65  Number  of  wronq  attempts  (b) 

66  Per  cent  correct  —  o/a  t  b 

Consequences 

67  Total  number  of  occeptoble  consequences 

68  Remoteness  score-hfqh  consequences 

Revision  II 

69  Total  number  of  ideas  retained  ( o ) 

70  Total  number  of  words  used  < b) 

71  Ratio  s  core  of  words/ideas  —  b/a 


Mean 

SD 

4.65 

1.08 

4.4  2 

1  .30 

4.74 

1  .05 

4.86 

1.02 

5.10 

1  .18 

4.54 

1  .31 

4.63 

1  .15 

4.8  0 

1.28 

4.73 

1.27 

4.52 

1.10 

4.58 

1.24 

4  84 

1 .27 

4.80 

1.19 

4.80 

1.01 

4.81 

1.27 

4.63 

1.31 

4.23 

1  .29 

1  1.26 

2.79 

6.44 

5.05 

9.87 

3.49 

9.25 

3.58 

10.64 

4.20 

10.68 

3.27 

7.66 

3.70 

9.48 

3.66 

8.94 

5.58 

10.44 

4.33 

5.73 

3.12 

11.64 

6.05 

14.16 

3.50 

12.56 

3.12 

12.53 

2.33 

11.27 

2.78 

13.07 

3.36 

6.87 

2.73 

11.87 

2.89 

15.07 

2.74 

11.46 

3.31 

9.48 

3.42 

7.35 

4.11 

1  1.01 

3.34 

14.21 

3.25 

12.1  3 

1  .47 

7.54 

3.26 

4.63 

2.58 

58.31 

18.94 

12.60 

4.56 

9.36 

4.31 

23.01 

3.88 

34.84 

6.28 

15.22 

2.27 

(Toble  continues  on  next  paqe) 
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TABLE  8  (Continued) 


Test  deri ved 

from  Nr  Description  of  score 

Visual  Imagery 

72  Tatol  number  af  correct  responses  (a) 

73  Wrong  ottempts  plus  skip-omits  { b ) 

7  4  Per  cent  correct  ol  thase  marked 

Word  Association 

75  Total  number  af  associated  wards  writton 

76  Total  number  ol  close  synonym-type  words 

77  Number  of  sets 

Minimum  Satisfaction  Scale 

78  Reading  skills  —  Self-rating  on  6-paint  scale 

79  Speaking  sk ills  —  Self-rat ing  an  6-point  scale 

80  Listening  skills  —  Self-rating  an  6-paint  scale 

81  Writing  skill  s  —  Self -rating  on  6-point  scale 

82  Quantity  of  wark  aut put  —  Se lf-rating  an  6-paint  scale 

83  Being  well  liked  —  Self-rating  on  6-paint  scale 

84  Administrative  advancements  —  Self-rating  on  6-point  scale 

85  Being  well  known  —  Self -rati  ng  on  6-point  scale 

86  Quontity  of  reports  —  Self -rat  ing  an  5-point  scale 

87  Theoretical  cantnbut  ions  —  Self  -rat  ing  an  5-paint  scale 

88  Experimental  contributions  —  Self-rating  on  5-point  scale 

89  Level  of  original  wark  —  Self -rat  ing  on  5-point  scale 

Pertinent  Question 

90  Total  acceptable  questions 

Apparatus  Test 

91  Total  acceptable  modifications 
Motivational  Analysis  Test 

92  Fear  — Number  right 

93  Sex  Number  right 

94  Assertion  —  Number  right 

95  Self-sentiment  —  Number  right 

96  Wife-sweetheart  —  Number  right 

97  Sadism  —  Number  right 

98  Career  —  Number  right 

99  Super  ega—  Number  right 

100  Parents-ham e  —  Numb er  right 

101  Narcism-comfart  —  Number  right 

Check  List  far  Scientists 

102  States  af  at  tent )  an  —  Mum  ber  right 

103  States  of  feeling  —  Number  right 

Undergraduate  College  Record 

101  Grade-Paint  Average 

Biaqr aphi  ;al  Inventory  Blank 

105  Criterion  A-empincal  key  —  Developmental  history 

106  Criterion  A-empirical  key  —Parents  and  family  life 

«  107  Criterion  A-empincal  key  —Academic  history 

108  Criterion  A-empirical  key  —  Adult  life  and  interests 

109  Criterion  A-empirical  key—  Totol  score 

110  Criterion  F-empirical  key  —  Developmental  history 

111  Criterion  F-emp irical  k ey  —  Parents  and  family  life 

112  Criterion  F-empiricol  key  Academic  history 

113  Criterion  F-empiricol  key  Adult  life  and  interest 

114  Criterion  F-empirical  key  — Total  scare 

115  Criterion  H-empirical  key  —  Developmental  history 

116  Criterion  H-empirical  key  — Parents  and  family  life 

117  Criterion  H-empirical  key  —  Academic  history 


Me  on  SD 


12.92  4.36 

4.13  2.82 

68.70  22.77 

27.50  9.80 

13.68  4.1  2 

15.51  3.24 


4.40  0.96 

4.51  1.02 

4.26  1.01 

4.54  0.98 

4.37  1.00 

3.98  0.98 

3.72  1.36 

3.93  1.05 

2.93  1.12 

3.08  1.19 

3.31  1.09 

3.70  1.07 

13.61  2.21 

10.77  3.73 

11.60  2.42 

6.87  2.22 

6.67  1.83 

40.73  5.69 

10.36  2.42 

7.90  2.70 

12.02  2.31 

18.10  3.53 

9.64  3.06 

9.89  2.48 

1  5.55  5.87 

2.92  1.99 

5.31  1.03 


50.81  4.60 

51.12  4.81 

53.91  6.85 

57.43  12.96 

71.11  8.20 

50.36  3.17 

50.10  4.24 

46.98  4.38 

52.21  10.12 

66.44  5.98 

50.58  2.92 

51  .94  4.21 

48.25  4.20 


(Table  continues  an  next  page) 
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TABLE  8  (Confjmjec/) 


Test  derived 

from  Nr  Description  of  score 

Biographical  Inventory  Blank  (CuntinUtui) 

118  Criterion  lf-empirical  key  —  Adult  life  and  interests 

119  Criterion  lf-empirical  key  — Total  score 

120  Criterion  D-empirical  key  —  Developmental  history 

121  Criterion  D-empirical  key  —  Parents  ard  family  life 

122  Criterion  D-empirical  key  —  Academic  history 

123  Criterion  D-empirical  key— Adult  life  and  interests 

124  Criterion  D-empirical  key  —  Tota  1  score 

125  Criterion  16-empirical  key  —  Developmental  history 

126  Criterion  1C  empirical  key  Parents  and  family  life 

127  Criterion  16-empirical  key—  Academic  history 

128  Criterion  16-empirical  key  —  Adult  life  and  interests 

129  Criterion  16-empirical  key— Total  score 

130  Criterion  L-empirical  key  —  Developmental  history 

131  Criterion  L-empirical  key  —  Parents  and  family  life 

132  Criterion  L-empirical  key  —  Academic  history 

133  Criterion  L-empirical  key— Adult  life  and  interests 

134  Criterion  L-empirical  k  ey  —  Total  score 

1  35  Professional  self-confidence  —  A  priori  key-positive  responses  only 

136  Emotional  restraint  —  A  priori  key-positive  responses  only 

137  Low  sociability  —  A  priori  key-positive  responses  only 

138  High  self-suf f iciency  —  A  priori  key-positive  responses  only 
1  39  Inner  directedness  —  A  priori  key-positive  responses  only 

140  Dedication  to  work  —  A  priori  key-positive  responses  only 

141  Liking  to  think  —A  priori  key-positive  responses  only 

142  Intellectual  thoroughness  A  prion  key-positive  responses  only 
1  43  Social  desirability  —  A  oriori  key-positive  responses  only 

144  Self-reported  ocademic  level  —  A ve rage  on  pertinent  self-roting  items 

145  Modal  response  —  Number  of  times  average  response  chosen 

146  General  activity  level 

Profile  Matching 

1  47  Supervisor's  rating  of  scientist  ond  his  job  —  Total 

148  Supervisor's  rating  of  scientist  and  his  job— Shape 

149  Supervisor's  rating  vs.  scientist's  rahnq  of  scie ntist  —  Total 

150  Supervisor's  rating  vs.  scientist's  rating  of  his  job— Scatter  and  shape 


Meon  SD 


49.96  9.34 

66.93  5.71 

48.94  3.30 

49.63  4.29 

46.69  2.78 

47.93  9.27 

64.44  5.40 

50.51  3.49 

49.93  6.73 

49.89  3.42 

63.12  7.55 

67.64  5.39 

50.58  3.13 

49.72  4.08 

50.58  4.42 

48.72  8.87 

66.44  5.77 

4.58  2.1  3 

2.33  1.52 

4.66  2.40 

3.78  2.24 

3.24  1.61 

5.11  1.59 

4.97  2.40 

2.26  1.51 

4.19  2.17 

28.01  4.70 

6.82  2.94 

17.09  11.27 

7.13  1.86 

1.08  0.26 
6.42  1.92 

2.72  0.50 
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TABLE  9.  Comporison  of  Main  Sample  with  Turnover  Sample 

-  ,  Main  Sample  (107)  Resignations  (28) 

Criterion  - - - - 


Score  Nr 

Name 

M 

SO 

M 

SD 

t* 

1 

Quantity  or  productivity  in  written  work 

51.02 

7.05 

49.46 

5.04 

1.33 

2 

Recent  publications 

1.01 

1.94 

.89 

1.26 

.40 

3 

Quality  (without  originality)  of  research 

reports 

49.69 

8.26 

52.43 

7.15 

-1.76 

4 

Originality  of  writ**»n  work 

21.49 

3.88 

21.04 

1.05 

1.07 

S 

Scientific  and  professional  society  mem¬ 

bership 

11.69 

9.38 

12.26 

8.95 

-.31 

6 

Actual  work  output,  as  judged  by  peers, 

supervisor,  and  lab  chief 

49.78 

6.71 

48.04 

6.68 

1.22 

7 

Creativity  rating  by  lab  chief 

16.51 

3.34 

17.04 

1.76 

-1.1  5 

8 

Overall  performance 

69.75 

15.61 

67.43 

1  5.78 

.70 

9 

Likableness 

50.73 

6.11 

46.86 

10.21 

.96 

1  0 

Visibility 

1*0. 1  4 

7.24 

49.25 

5.86 

.68 

1  1 

Recognition  for  organizational  contributions 

49.18 

4.53 

50.46 

6.76 

-.95 

12 

Status -seeking,  "organizational-man*' 

tendencies 

11.56 

5.10 

11.93 

4.88 

-.36 

1  3 

Current  organizational  status 

49.37 

3.01 

48.32 

3.1  3 

1.57 

1  4 

Contract  monitoring  load 

1.85 

3.37 

.57 

.82 

3.55 

15 

Peer  Ranking 

52.58 

1  5.71 

49.96 

1  5.1  2 

.89 

16 

Supervisory  rating  of  drive-resourcefulness 

8.25 

2.21 

7.89 

2.27 

.77 

17 

Supervisory  rating  of  creation 

3. 69 

1.20 

3.71 

.99 

.09 

18 

Control  score  of  age 

35.76 

7.65 

31.57 

6  59 

2.89 

1  9 

Control  score  of  work,  years 

10.61 

5.92 

7.96 

4.81 

2.48 

20 

Control  score  of  months,  Center 

6.08 

4.04 

4.79 

3.07 

1.87 

In  this  situation,  t^c  1.96;  tgj  2.62. 


TABLE  10.  Comparison  of  Main  Sample  with 
Those  Wha  Did  Not  Completely  Caaperote 


Criterion 


Main  Sample  (107)  Less  Cooperati  vc 


Score  Nr 

Nam  c 

M 

SD 

M 

SD 

t* 

1 

Quantity  or  productivity  in  written  woik 

51.02 

7.05 

48.80 

3.47 

2.64 

2 

Recent  publications 

1.01 

1.94 

.71 

1  .74 

.97 

3 

Quality  (without  originality)  of  research 
reports 

49.69 

8.26 

49.86 

4.07 

-.1  7 

4 

Originality  of  written  work 

21.49 

3.88 

20.92 

.a6 

1.42 

5 

Scientific  and  professional  society  mem¬ 
bership 

11.69 

9.38 

8.67 

6.99 

2.34 

6 

Actual  work  output,  as  judged  by  peers, 
supervisor,  and  lab  chief 

49.78 

6.71 

49.61 

5.75 

.16 

7 

Creativity  rating  by  lab  chief 

16.51 

3.34 

16.14 

3.13 

.67 

8 

Overall  performance 

69.75 

1  5.61 

69.39 

14.39 

.15 

9 

Likableness 

50.73 

6.11 

49.26 

4.98 

1.60 

10 

Visibility 

50.14 

7.24 

49.82 

5.29 

.31 

1 1 

Recognition  for  organizational  contributions 

49.18 

4.53 

51.16 

7.46 

-1 .72 

12 

Status -seeking,  **  organ  iz  at  ionol-mon** 
tendencie  s 

11.56 

5.10 

10.26 

4.6  5 

1.57 

1  3 

Current  organizational  status 

49.37 

3.01 

48.94 

3.24 

.74 

14 

Contract  monitoring  load 

1.85 

3.37 

1.61 

1.63 

.60 

15 

Peer  ranking 

52.58 

15.71 

53.45 

15.67 

-.32 

16 

Supervisory  rating  of  drive-resourcefulness 

8.25 

2.21 

8.61 

1.98 

-1.03 

1  7 

Supervisory  rating  of  creation 

3.6  9 

1.20 

3.53 

1.09 

.84 

18 

Control  score  of  age 

35.76 

7.65 

36.59 

5.72 

-.75 

1  9 

Control  score  of  work,  years 

10.61 

5.92 

12.12 

5.92 

-1.48 

20 

Control  score  of  months,  Center 

6.08 

4.04 

7.67 

3.84 

-2.37 

In  this  situation,  tQ^  1.98,*  tQ1  2.62. 
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TABLE  11.  Validity  Coefficients  for  17  Criterio 


Criterion  Score 


Test 

Score 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

21 

30 

15 

-05 

-01 

20 

27 

22 

36 

06 

02 

03 

-05 

07 

-09 

26 

32 

25 

22 

16 

04 

-06 

05 

19 

18 

01 

19 

19 

03 

09 

-03 

19 

-02 

19 

02 

16 

23 

02 

14 

-06 

06 

18 

27 

10 

24 

12 

04 

00 

-07 

23 

-01 

26 

20 

17 

24 

12 

10 

02 

22 

30 

34 

19 

33 

31 

14 

15 

-02 

35 

07 

31 

1  8 

22 

25 

07 

-02 

09 

-09 

20 

07 

03 

-03 

01 

-01 

09 

-21 

05 

-1  3 

05 

-10 

-1  2 

26 

19 

07 

03 

-02 

16 

11 

15 

14 

02 

-0  2 

-0  2 

-13 

13 

13 

12 

18 

04 

27 

17 

06 

-08 

09 

31 

28 

20 

24 

30 

20 

13 

-10 

21 

00 

26 

21 

16 

28 

29 

02 

03 

14 

33 

27 

18 

16 

26 

06 

12 

-1 1 

19 

-09 

26 

1  4 

08 

29 

11 

00 

01 

09 

33 

25 

07 

13 

18 

04 

09 

-03 

23 

02 

24 

02 

-01 

30 

00 

11 

-0  5 

-0  3 

16 

18 

1  1 

12 

08 

-01 

03 

-06 

01 

-10 

18 

15 

04 

31 

04 

05 

08 

10 

24 

35 

06 

22 

33 

22 

1  3 

-02 

25 

13 

37 

08 

1  0 

32 

10 

10 

-07 

06 

27 

13 

05 

01 

10 

02 

-0  4 

-15 

09 

-15 

1  3 

00 

-06 

33 

28 

07 

-07 

1  2 

24 

46 

08 

24 

1  5 

11 

1  5 

-10 

30 

01 

46 

24 

16 

34 

27 

1  7 

-03 

15 

27 

38 

13 

34 

41 

20 

19 

01 

39 

09 

36 

20 

23 

35 

-16 

-01 

1  4 

-04 

24 

15 

08 

06 

07 

05 

18 

08 

1 1 

03 

12 

01 

-06 

36 

1  4 

04 

-01 

00 

36 

25 

16 

25 

31 

16 

1  9 

03 

41 

12 

20 

16 

18 

37 

34 

07 

-12 

22 

41 

31 

2  1 

29 

34 

20 

11 

-10 

33 

-02 

30 

22 

23 

38 

06 

02 

06 

02 

-0  2 

06 

02 

04 

-03 

-08 

1  0 

09 

03 

01 

05 

-0  4 

06 

39 

08 

06 

-07 

-0  3 

-0  8 

-10 

09 

05 

-06 

07 

13 

03 

05 

05 

-11 

03 

10 

40 

01 

04 

-10 

-06 

00 

-12 

-0  7 

-03 

00 

-11 

06 

07 

01 

08 

-1 1 

-07 

-10 

41 

-1  1 

-1  9 

04 

-06 

-01 

06 

-09 

-04 

04 

-01 

-1  1 

-16 

-0  5 

01 

04 

-05 

-09 

42 

05 

-24 

17 

02 

-04 

03 

-03 

-12 

21 

04 

11 

08 

18 

07 

01 

-1  8 

-04 

43 

15 

28 

-04 

02 

16 

-06 

12 

-04 

-08 

-11 

-02 

02 

-09 

03 

-04 

-03 

-0  5 

44 

-0  9 

-01 

-01 

-2  5 

-10 

-14 

00 

-1 1 

-13 

-20 

05 

07 

-14 

-12 

-12 

-1 1 

-11 

45 

-19 

-13 

00 

-18 

-08 

-05 

-07 

-01 

-12 

-08 

-01 

04 

-17 

06 

-04 

-01 

01 

46 

-03 

07 

10 

-0  4 

10 

-03 

01 

-04 

03 

04 

02 

11 

00 

23 

-07 

-09 

04 

47 

23 

00 

02 

1  4 

07 

1 1 

19 

-03 

16 

10 

-07 

-20 

19 

-02 

09 

-05 

08 

48 

-1  9 

-01 

04 

-0  9 

-18 

-13 

-12 

-09 

-20 

05 

-0  5 

06 

-28 

-1  1 

-12 

-13 

-02 

49 

-10 

-0  9 

06 

-05 

-06 

-08 

-06 

-26 

-13 

-07 

-12 

-10 

-16 

03 

-05 

-23 

-28 

50 

07 

-07 

-17 

08 

06 

-05 

07 

-19 

-06 

-11 

-12 

-07 

-08 

-0  5 

-08 

-14 

-18 

51 

-04 

03 

04 

05 

07 

01 

01 

-02 

05 

11 

1  1 

-04 

09 

05 

01 

-03 

-01 

52 

04 

-27 

03 

-1  4 

-14 

06 

-1  1 

-06 

08 

-06 

05 

-07 

1 1 

-05 

07 

-13 

07 

53 

1  3 

04 

1  1 

-17 

23 

12 

19 

1  5 

05 

03 

09 

02 

12 

16 

18 

11 

12 

54 

-1  1 

12 

07 

03 

1  5 

19 

16 

10 

04 

04 

-03 

05 

02 

22 

1  8 

21 

-06 

55 

-0  7 

12 

-21 

00 

05 

-11 

-15 

-0  8 

-07 

00 

-0  3 

05 

-0  3 

-08 

-04 

-05 

-05 

56 

-07 

-08 

14 

-06 

16 

01 

-04 

-13 

-0  5 

03 

03 

-02 

09 

12 

00 

-10 

-17 

57 

-07 

03 

03 

-1  1 

05 

12 

-10 

01 

-13 

-06 

-18 

-04 

-01 

-02 

1  4 

06 

-10 

58 

00 

-02 

-04 

-03 

-07 

-2  0 

-06 

-20 

-08 

-01 

2  2 

06 

-13 

-21 

-12 

-22 

-01 

59 

05 

-06 

02 

04 

12 

1  1 

00 

1  4 

28 

-01 

09 

03 

19 

-06 

09 

08 

07 

60 

-1  1 

12 

17 

-0  4 

21 

1  5 

10 

09 

10 

04 

05 

01 

15 

1  3 

13 

05 

1  1 

61 

01 

05 

05 

21 

09 

03 

03 

-05 

02 

08 

00 

12 

-08 

-15 

03 

00 

-11 

62 

14 

04 

12 

05 

04 

03 

11 

-03 

1  5 

-01 

1  3 

04 

00 

03 

-0  4 

-0  5 

05 

63 

-02 

15 

-16 

-17 

00 

-07 

-07 

-05 

-15 

-01 

03 

-11 

-10 

-09 

-01 

-05 

-03 

64 

18 

03 

-1  2 

17 

04 

1  6 

07 

12 

20 

-08 

13 

01 

1 1 

14 

11 

05 

06 

65 

-09 

05 

12 

-18 

-13 

-07 

09 

03 

-05 

-02 

-12 

05 

-05 

-08 

-09 

01 

08 

66 

1  7 

-15 

1  8 

06 

-03 

07 

02 

05 

15 

-08 

10 

-10 

07 

-01 

02 

00 

01 

67 

-01 

06 

08 

-1  1 

06 

-1  5 

-14 

-14 

14 

-06 

06 

-02 

09 

14 

-15 

-16 

-12 

68 

-07 

03 

06 

-0  5 

05 

-1  4 

-07 

-09 

17 

-05 

05 

07 

13 

08 

-14 

-14 

-02 

69 

08 

-19 

1  0 

-08 

09 

09 

01 

02 

1  4 

-10 

06 

00 

17 

-01 

12 

11 

-01 

70 

1  5 

-13 

1  4 

-06 

05 

-01 

00 

-0  5 

08 

-18 

-05 

-11 

09 

-12 

04 

05 

-04 

71 

03 

05 

08 

00 

-10 

-04 

-08 

-06 

-07 

-12 

-16 

-08 

-10 

-12 

-01 

-0  5 

-0  3 

72 

1  4 

01 

07 

11 

08 

04 

09 

18 

18 

01 

06 

07 

02 

-08 

03 

01 

20 

73 

-0  8 

-11 

01 

-1 1 

-16 

-04 

-0  5 

-05 

-10 

-13 

-20 

01 

-2  0 

-05 

-0  2 

02 

-04 

74 

09 

11 

-03 

15 

02 

10 

00 

07 

28 

18 

-07 

03 

20 

18 

08 

05 

06 

75 

-09 

03 

-01 

00 

18 

01 

-11 

-01 

16 

01 

05 

07 

10 

-01 

03 

-10 

04 

76 

03 

01 

06 

16 

-01 

-06 

-05 

-03 

11 

08 

13 

12 

17 

01 

-06 

-07 

04 

No»e.~tQ^  .19;  ~  .25  (N  107);  decimal  points  omitted. 

(Table  continues  on  next  page) 
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TABLE  11  (Continued) 


Criterion  Score 


Test 

Score 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

77 

-0  4 

02 

17 

14 

-01 

11 

00 

-03 

11 

23 

01 

-04 

10 

04 

07 

01 

-04 

78 

13 

-11 

1  1 

05 

14 

13 

20 

14 

30 

-11 

05 

-03 

20 

-0  8 

16 

06 

09 

79 

23 

12 

-08 

-01 

18 

01 

12 

16 

27 

10 

07 

-03 

15 

00 

04 

06 

12 

80 

17 

-03 

09 

-04 

20 

11 

25 

07 

16 

00 

15 

-14 

17 

-08 

13 

00 

03 

81 

26 

02 

-02 

04 

37 

18 

18 

15 

46 

16 

18 

03 

47 

12 

17 

07 

09 

82 

30 

03 

1  5 

00 

27 

08 

08 

08 

16 

12 

19 

-23 

06 

-07 

10 

05 

00 

83 

15 

1  7 

-01 

-04 

17 

25 

10 

30 

03 

12 

12 

-09 

11 

04 

25 

17 

29 

84 

-0  5 

12 

05 

-12 

01 

-04 

-0  3 

19 

01 

11 

12 

05 

04 

04 

03 

12 

14 

85 

13 

05 

04 

-03 

29 

00 

-01 

23 

24 

17 

12 

-11 

25 

02 

06 

09 

17 

86 

41 

1  7 

01 

20 

33 

1  3 

10 

27 

24 

17 

18 

-04 

21 

-02 

11 

17 

26 

87 

38 

07 

-10 

11 

39 

19 

13 

30 

19 

17 

16 

-12 

33 

-02 

15 

17 

31 

88 

05 

-01 

1  3 

04 

09 

01 

-08 

-06 

-0  5 

11 

17 

-11 

-09 

-12 

03 

-01 

-06 

89 

28 

07 

07 

15 

35 

20 

10 

17 

29 

16 

14 

-0  8 

20 

-05 

20 

05 

21 

90 

01 

01 

00 

00 

-01 

02 

-01 

05 

1  1 

08 

-01 

04 

04 

-02 

04 

04 

12 

91 

05 

12 

03 

06 

2(5 

1  3 

08 

03 

24 

10 

01 

01 

20 

03 

09 

00 

11 

92 

09 

10 

1  4 

10 

-09 

13 

18 

06 

02 

06 

-03 

-01 

09 

10 

13 

90 

06 

93 

01 

07 

01 

04 

-02 

-04 

04 

-01 

-05 

-01 

-09 

-1C 

-03 

-04 

-05 

03 

-08 

94 

-15 

-09 

05 

-1 1 

-06 

-27 

-19 

-13 

-11 

-12 

-16 

01 

-12 

-05 

-21 

-13 

-14 

95 

12 

-0  8- 

20 

05 

06 

07 

18 

-0  4 

20 

10 

05 

21 

1  2 

12 

10 

-0  8 

07 

96 

06 

-07 

-04 

02 

-01 

18 

-01 

-02 

09 

14 

24 

07 

20 

05 

16 

-01 

-01 

97 

15 

08 

-12 

19 

-0  4 

13 

-07 

03 

-12 

-05 

02 

-02 

-09 

10 

12 

07 

03 

98 

18 

08 

-07 

-01 

08 

1  3 

04 

10 

09 

17 

02 

-10 

04 

33 

14 

15 

06 

99 

-1  1 

01 

08 

00 

-01 

07 

-01 

-06 

-01 

12 

09 

21 

11 

15 

03 

-07 

-03 

100 

-04 

00 

08 

-16 

01 

01 

-13 

08 

-22 

-10 

06 

02 

-0  5 

10 

-05 

15 

04 

1  01 

05 

-01 

06 

08 

03 

00 

24 

06 

30 

01 

02 

14 

12 

14 

00 

-01 

11 

102 

01 

12 

-06 

-06 

06 

11 

09 

11 

04 

01 

12 

-18 

-04 

09 

11 

07 

05 

103 

11 

01 

-18 

-09 

13 

08 

-08 

04 

12 

23 

37 

-12 

17 

-02 

09 

-04 

-03 

104 

27 

10 

-01 

-02 

08 

16 

21 

13 

13 

07 

08 

-06 

19 

02 

19 

04 

12 

105 

64 

06 

12 

17 

25 

18 

27 

07 

34 

1  1 

12 

-07 

29 

25 

15 

-01 

08 

106 

70 

03 

1  5 

04 

23 

10 

15 

1  5 

24 

04 

11 

1  5 

1  1 

09 

1 1 

-02 

16 

1  07 

66 

-01 

05 

16 

20 

24 

29 

13 

24 

15 

28 

-02 

26 

04 

26 

-05 

21 

108 

80 

-03 

09 

24 

27 

24 

31 

24 

41 

10 

13 

08 

29 

05 

25 

15 

26 

109 

87 

00 

12 

21 

29 

25 

32 

21 

39 

13 

19 

05 

30 

10 

25 

06 

24 

110 

20 

19 

-01 

32 

26 

63 

33 

37 

26 

32 

17 

-04 

23 

28 

62 

38 

20 

1  1  1 

20 

14 

-17 

26 

30 

67 

19 

41 

32 

35 

21 

-06 

26 

26 

66 

42 

25 

1  12 

29 

1  5 

-02 

19 

34 

57 

26 

46 

32 

32 

30 

-12 

41 

09 

57 

30 

36 

1  1  3 

30 

17 

08 

28 

25 

76 

32 

46 

47 

28 

29 

05 

46 

25 

75 

35 

28 

1  1  4 

33 

21 

00 

33 

35 

83 

34 

55 

47 

38 

32 

00 

48 

29 

82 

45 

35 

1  1  5 

06 

16 

-04 

30 

20 

44 

32 

66 

19 

29 

12 

08 

22 

07 

37 

58 

54 

116 

26 

23 

-10 

09 

27 

44 

30 

64 

32 

18 

1  7 

11 

24 

14 

35 

48 

55 

117 

21 

24 

-04 

18 

27 

43 

2  3 

64 

29 

24 

26 

-01 

32 

14 

39 

44 

52 

1  18 

26 

25 

08 

22 

26 

51 

39 

76 

46 

29 

19 

13 

33 

16 

45 

66 

62 

1  1  9 

26 

28 

-01 

24 

3  1 

57 

40 

84 

44 

32 

23 

11 

36 

16 

49 

69 

69 

120 

22 

27 

-05 

64 

28 

30 

22 

12 

20 

37 

14 

-07 

10 

18 

31 

17 

12 

121 

09 

08 

-1  1 

60 

06 

27 

14 

04 

12 

17 

08 

-0  3 

00 

14 

28 

12 

02 

122 

30 

10 

-02 

60 

1  1 

23 

11 

1  9 

18 

23 

16 

05 

1 1 

08 

22 

14 

20 

1  23 

24 

16 

03 

81 

1  9 

32 

28 

15 

31 

20 

-06 

02 

17 

15 

28 

23 

10 

124 

2  5 

19 

-0  3 

85 

19 

36 

25 

15 

27 

27 

04 

00 

1  4 

1  8 

33 

21 

12 

125 

05 

17 

-07 

20 

75 

38 

37 

54 

19 

21 

05 

10 

15 

15 

31 

72 

40 

1  26 

00 

32 

-10 

08 

09 

18 

01 

25 

07 

09 

07 

-08 

-01 

10 

13 

45 

15 

1  27 

08 

25 

04 

09 

1  2 

40 

08 

49 

18 

17 

20 

-07 

27 

14 

33 

63 

2  7 

128 

1  6 

38 

01 

29 

2  3 

42 

29 

65 

22 

20 

05 

05 

25 

21 

37 

83 

44 

1  29 

1  3 

34 

-04 

24 

23 

47 

30 

66 

24 

23 

12 

02 

22 

22 

40 

88 

43 

130 

-06 

-08 

00 

07 

1  2 

11 

08 

10 

10 

-0  4 

-0  4 

67 

1  1 

11 

07 

12 

15 

131 

14 

-05 

03 

01 

0  5 

-03 

14 

10 

22 

00 

-0  6 

60 

1  3 

06 

-03 

02 

21 

132  - 

-03 

-12 

06 

02 

13 

-02 

18 

02 

12 

-0  6 

02 

65 

-02 

-01 

-03 

-08 

12 

133 

07 

-08 

03 

01 

1  0 

03 

11 

1  5 

25 

-10 

-01 

76 

14 

01 

01 

03 

16 

(Table  continues  on  next  page) 
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TABLE  11  (Continued) 


Criterion  Score 


Score 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

134 

06 

-09 

04 

03 

1 1 

02 

15 

12 

24 

-07 

-01 

83 

1  2 

05 

00 

01 

19 

135 

41 

19 

-01 

14 

19 

32 

16 

26 

23 

30 

19 

-09 

36 

16 

33 

14 

23 

136 

04 

03 

-09 

09 

16 

23 

23 

18 

26 

05 

18 

-03 

10 

10 

16 

1  8 

15 

137 

16 

07 

00 

17 

15 

11 

1  7 

34 

17 

16 

21 

05 

14 

00 

06 

35 

32 

138 

28 

o: 

00 

16 

1  4 

23 

25 

18 

34 

17 

29 

09 

23 

-01 

19 

10 

25 

139 

34 

-01 

-03 

23 

27 

19 

23 

27 

31 

13 

13 

20 

27 

19 

18 

22 

24 

1  40 

31 

03 

-03 

09 

29 

15 

20 

09 

03 

09 

04 

-05 

11 

13 

10 

-03 

19 

141 

43 

-10 

17 

14 

22 

13 

30 

11 

25 

02 

12 

15 

19 

-02 

12 

-07 

10 

142 

32 

1  3 

02 

03 

21 

17 

10 

22 

27 

1  0 

02 

-08 

37 

08 

19 

08 

22 

143 

-25 

-1  4 

20 

-17 

-06 

-1  3 

-17 

-21 

-18 

-01 

-03 

-1  5 

-10 

-09 

-10 

-24 

-18 

144 

17 

00 

09 

-05 

28 

15 

06 

21 

08 

1  0 

04 

01 

22 

-02 

1  9 

02 

24 

145 

-26 

-08 

00 

-13 

-34 

-1  1 

-23 

-26 

-3  3 

-15 

-1  3 

-10 

-38 

-1  9 

-06 

-1  2 

-24 

146 

02 

04 

12 

“03 

-12 

-07 

14 

09 

01 

-09 

-03 

08 

-06 

-10 

-01 

-0  3 

04 

147 

12 

1  6 

-12 

05 

27 

38 

30 

71 

37 

31 

32 

04 

30 

00 

34 

53 

67 

148 

15 

11 

04 

12 

1C 

•  08 

-02 

-07 

15 

25 

36 

-09 

16 

1  6 

-10 

-1  7 

-02 

149 

-04 

-1  0 

01 

02 

-23 

-2  3 

-2  7 

42 

-1  9 

-08 

-1  5 

-06 

-08 

-01 

-1  9 

-38 

-31 

150 

00 

14 

-08 

-3  5 

-32 

26 

-13 

-1  7 

01 

-18 

1  1 

-08 

-03 

-34 

-22 

-2  5 

-09 
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APPENDIX  B 


RATIONALE  FOR  CONSTRUCTION  OF  ONE  NEW  MEASURING  INSTRUMENT: 

THE  CREATIVE  PROCESS  CHECK  LIST* 

It  seems  appropriate  to  present  an  illustration  of  problems  in  the  development  of  new  meas¬ 
uring  instruments  in  the  important  area  of  scientific  talent,  especially  of  the  creative  type.  The 
illustration  chosen  was  the  Creative  Process  Check  List  with  its  two  subscores  from  the  states- 
of-attention  check  list  and  the  states-of-fe<>ling  check  list. 

Our  research  in  several  areas  has  demonstrated  the  dependability  of  instruments  of  self- 
report,  including  check  lists,  as  a  means  of  measuring  ability  and  predicting  success.  It  occurred 
to  us  that  the  introspection  of  scientists  upon  their  processes  of  production  might  yield  useful  and 
perhaps  unexpected  insights  if  it  could  be  projected  objectively  for  comparative  study. 

It  is  well  known  that  many  creative  people,  scientists  and  others,  have  recorded  their  sub¬ 
jective  experience  in  production,  making  statements  exact  enough  to  yield  insight  into  their  psycho¬ 
logical  processes  and  to  allow  some  comparison  of  these  processes  with  one  another.  An  example 
of  a  body  of  statements  of  this  kind,  made  by  mathematicians,  may  be  seen  in  the  book  The  Psy¬ 
chology  ui  invention  in  the  Mathematical  Field,  by  Jacques  Hadamard.  The  interest  of  first-rate 
minds  in  making  and  comparing  records  of  subjective  experience  in  production  is  indisputable. 

The  value  of  such  studies  can  no  more  be  denied  than  it  can  be  estimated  with  finality.  But  the 
available  records  have  two  shortcomings:  there  is  in  the  first  place  too  little  material ,  and  in  the 
second  place  the  records  are  very  diverse  in  terminology,  in  mode  of  presentation,  and  in  scope. 

Our  check  list,  an  instrument  by  means  of  which  some  of  the  subjective  experience  of  production 
could  be  recorded  in  one  linguistic  schema,  would  compel  large  numbers  of  subjects  to  use  a 
common  language  in  reporting  in  the  same  fashion  on  the  same  matters,  and  it  would  enable  us  to 
project  reported  data  as  a  complex  of  measurable  dimensions. 

In  determining  what  information  to  call  for,  we  were  guided  by  Ghiselin's  (1958)  criterion  of 
creativity:  the  idea  that  creativity  is  an  "origination  of  significant  order  in  the  subjective  sphere," 
the  shaping  for  the  first  time  of  a  part  of  the  accepted  "universe  of  meaninq"  in  terms  of  which 
we  understand  our  world  and  ourselves.  We  saw  as  crucial  the  act  of  producing  such  fresh  insight, 
of  forming  and  bringing  into  focal  attention  a  new  configuration.  It  was  observed,  moreover,  that 
the  best  introspective  explorations  of  presumably  creative  production  were  much  concerned  with 
the  character  of  the  producer's  attention  and  the  quality  of  his  feeling.  We  designed  the  check  list 
to  supply  information  about  the  states  of  attention  and  qualities  of  feeling  which  the  worker  had 
experienced  at  three  critical  moments  of  production:  immediately  before,  during,  and  immediately 
after  the  formation  of  a  new  configuration  of  insight. 

A  basic  problem  in  making  the  check  list  was  semantic.  The  items  of  the  check  list  must 
refer  to  complex  and  elusive  subjective  states  by  means  of  words  lacking  the  absolute  exactitude 
of  scientifically  limited  terminology.  The  problem  was  to  present  to  the  subjects  a  usable  vocabu¬ 
lary,  controllably  limited  in  size  yet  sufficient  in  scope  and  precision  to  supply  a  language  for 
reporting  the  phenomena  we  hoped  to  use  in  research. 

In  building  the  check  list,  terms  were  chosen  in  an  effort,  first,  to  approximate  the  vocabu¬ 
lary  employed  by  a  variety  of  intelligent,  experienced  workers  in  various  fields  to  indicate  the 


*  Prepared  by  Hrcwster  Ghisehn. 


51 


CHECK  LIST  FOR  SCIENTISTS  STATES  OF  ATTENTION  AND  FEELING  DURING  PROBLEM  SOLVING  OR  CREATIVE  WORK 


Central  Instructions:  It  la  often  reported  that  people  have  varied  personal  feelings  and  experiences  during  different  stage*  of  problem 
solving  or  creative  acta,  la  order  to  Ideotlfy  some  of  the  poasible  feelings  and  experiences  which  scientists  have  while  engaged  In 
their  work,  we  would  like  you  to  check  the  adjectives  in  the  two  lists  below  which  best  describe  your  attention  and  feeling  during  cre¬ 
ative  acts  or  insightful  mental  experience. 

In  List  A  are  some  adjectives  wh  h  might  pertain  to  your  attention  during  such  acts.  In  List  B  are  some  other  adjectives  which 
might  pe rtaln  to  your  feelings  during  I  acts  What  Is  sought  Is  your  selection  of  adjectives  that  can  best  serve  to  report  the  states 
of  your  ention  (List  A)  immediately  before,  during,  and  after  your  action  of  forming  a  new  insight  or  new  configuration  of  any  kind; 
and  the  state  of  your  feeling  (List  B)  immediately  before,  during,  and  after  your  action  or  forming  a  new  insight  or  new  configuration 

In  completing  the  following  checklist*,  you  are  to  consider  the  whole  group  of  adjectives  in  each  list.  If  an  adjective  applies 
to  at  least  some  extent  at  the  particular  stage  of  the  process  under  consideration,  give  it  a  single  check.  Otherwise  do  not  mark  the 
adjective  at  all  In  marking  each  checklist  column,  indicate  what  you  feel  to  be1  true  at  each  stage,  even  if  at  times  you  may  seem  to 
be  inconsistent  in  the  pattern  of  adjectives  you  mark. 


You  may  check  any  or  all  of  the  adjectives  If  they  pertain  to  your  experiences  Begin  with  List  A  When  you  hsve  finished  the 
task  described  there  please  go  on  to  List  B  There  is  no  time  limit  for  this  exerclse--merely  think  shout  your  experiences  and  try  to 
mark  the  adjectives  which  seem  appropriate  to  you  for  each  given  condition  within  each  list . 


Check  those  adjectives  in  the  list  below  which  best  des¬ 
cribe  the  various  states  of  ATTENTION  which  you  ex¬ 
perience  ( l  )  immediately  before  you  grasp  a  new  insight 
(solution  to  a  problem,  etc .)  (2)  during  the  moment  (s) 
when  you  are  grasping  (shaping)  a  new  insight,  etc  and 
(3)  immedistcly  after  you  have  grasped  (shaped)  a  new 
insight,  etc.  Make  any  desired  additions  to  the  list  in  the 
blank  spaces  at  the  bottom.  Work  down  one  checklist 
column  at  a  time. 


(1) 

Immediately 

Before 


vague 

fixed 

fluid 

narrowed 

ranging 

inclusive 


expanded  (  ) 

wandering  (  ) 

orderly  (  ) 

scattered  (  ) 

c  lea  r  (  ) 

chaotic  (  ) 

contracted  (  ) 

muddled  (  ) 

wavering  (  ) 

intense  (  ) 

variable  (  ) 

faint  (  ) 

jumbled  (  ) 

fluctuating  (  ) 

dazzled  (  ) 

unconfigurated  (  ) 

arrested  (  ) 

configurated  (  ) 

sharp  (  ) 

characterless  (  ) 

dispersed  (  ) 

steady  (  ) 

scanning  (  ) 

searching  (  ) 


(3) 

Immediately 

After 


Check  those  adjectives  in  the  list  below  which  best  des¬ 
cribe  the  vsrious  states  of  FEELING  which  you  exi>erl- 
ence  ( 1)  immediately  hefore  you  grasp  a  new  insight 
(solution  to  a  problem,  etc.)  (2)  during  the  moment  (s) 
when  you  are  grasping  (shaping)  a  new  insight,  etc,  and 
(3)  immediately  after  you  have  grasped  (sha[>cd)  a  nt  w 
insight,  etc  Make  any  desired  additions  to  the  list  in 
the  hlank  spaces  at  the  bottom  Work  down  one  checklist 
column  at  a  time. 


(D 

immediately 
B<  fore 


sad 

cool 

inspirited 

exalted 

frightened 

restless 

satisfied 

feverish 

delighted 

ind:  fferent 

relieved 

impatient 

depressed 

eager 


confused 

lost 

ex pet  tant 

peaceful 

exhausted 

dulled 

emjny 

depleted 

enriched 

pressured 

pressed 

full 

disturbed 

troubled 

disorganized 


(3) 

Immediately 

After 


note  While  you  were  checking  the  *lwive  lists  what  sort  of  aet  (s)  did  you  have  in  mind?  Prohiem  somng(  ),  Creative* 
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states  of  attention  and  feeling  they  had  observed  in  themselves  in  the  course  of  their  productive 
activity,  and  secondly  to  represent  concepts  current  in  psychological  investigation  of  the  matter. 
Terms  obviously  flattering  or  denigrative  were  rejected  in  favor  of  neutral  equivalents.  It  was 
judged  best  not  to  afford  only  one  single  term  for  each  reference  that  we  anticipated,  for  the 
sufficiency  of  none  could  be  presupposed.  The  items  of  the  lists  include  such  words  as  vague 
(used  by  Einstein  and  many  others),  its  less  common  relative  muddled  (used  by  Whitehead),  and 
still  other  closely  related  terms.  Wo  excluded  the  more  highly  loaded  general  terms  that  are 
common  in  discussion  of  creativity,  such  as  "creative, "  "imaginative,"  "inspired,"  the  meaning 
of  which  is  largely  in  guestion  and  the  coverage  of  which  is  nearly  limitless,  but  we  provided  a 
body  of  terms  that  v/e  thought  could  represent  phenomena  these  terms  have  referred  to  in  responsi¬ 
ble  use.  (See  sample  farm.) 

In  preparing  to  scare  the  completed  check  list,  we  picked  out,  in  each  section  of  the  list, 
groups  oi  terms  indicative  of  discriminable  conditions  of  consciousness  which  we  judged  to  be 
distinctive  in  the  possible  modes  of  behavior,  whether  productive  of  configurations  or  unproductive, 
at  the  three  specified  stages  we  were  investigating.  For  the  purpose  of  interpreting  the  first  check 
list  on  states  of  attention,  we  discriminated  four  aspects  of  attention  that  we  considered  defini¬ 
tive.  These  we  described  as  "determinancy,"  or  configural  firmness  and  clarity;  "breadth,"  or 
inclusiveness  in  envisaging,  or  range  or  variety  in  the  activity  of  envisaging;  "flexibility,"  or 
freedom  of  movement  among  the  possible  objects  of  attention  or  from  one  state  of  attention  to 
another;  and  "indeterminacy,"  either  complete  absence  of  configural  forms  in  the  field  of  con¬ 
sciousness,  or,  in  an  imperfact  degree  of  such  configural  forms,  an  indefiniteness  in  their 
presentation.  V/e  next  devised  a  scale  of  ascending  values,  expressing  what  we  believed  to  be 
the  relative  fruitfulness  at  each  stage  of  production,  before,  during,  and  after,  of  the  behaviors 
designated  by  these  terms  singly  or  in  combination.  A  few  of  the  adjectives  in  the  check  list 
functioned  as  distractors  that  were  never  scored.  Other  adjectives,  scored  as  significant  in  one 
of  the  check  list  columns  for  one  stage  of  the  creative  process,  served  as  distractors  in  the  other 
two  check-list  columns  for  the  other  two  stages  of  the  creative  process.  Our  treatment  of  the 
second  check  list,  indicating  states  of  feeling,  was  similar. 

During  the  tentative  development  of  the  scoring  system  for  our  scale,  we  tested  its  validity 
against  the  records  of  certain  subjects  in  our  sample  of  scientists  whom  we  knew  best,  particularly 
those  whose  performance  was  least  in  doubt  and  whose  communication  in  interviews  with  us  about 
their  experiences  in  working  had  been  most  full,  frank,  and  illuminating.  We  did  not  change  our 
scale  to  conform  to  these  findings,  but  used  them  rather  to  coriect  our  theoretical  understanding, 
insofar  as  they  seemed  to  cast  light  upon  it.  Correction  of  the  scoring  system  for  the  scale  was 
made  upon  the  basis  of  such  enlargements  of  insiqht  as  we  could  stimulate  by  these  and  all  other 
means  at  our  disposal.  At  that  stage  we  felt  it  was  possible  for  the  originator  of  the  test  to  infer 
with  some  accuracy,  from  the  groups  of  items  checked  on  the  list,  the  characteristic  performance 
of  a  few  of  the  men  in  the  sample  whom  he  had  not  interviewed  and  whose  records  he  had  not 
studied,  but  who  were  so  well  known  to  another  member  of  the  research  team  that  he  could  esti¬ 
mate  the  success  of  the  inference. 
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APPENDIX  C 


SAMPLE  DATA  COLLECTION  FORMS 

Sample  Page:  Publication  and  Report  Rating  Scale 

Sample  Peer  Rating  Form 

Sample  Form:  Productivity  Ranking 

Sociometric  Rating  Sheet 

Sample  Page:  Personal  Data  Form 

Sample  Page:  Self-Rating  Form 

Satisfactory  Achievement  Scale 

Sample  Page:  Biographical  Information  Blank 
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Sample  Form:  Productivity  Ranking  ^ 

Productive 

RANKING  OF  EMPLOYEES  WITH 
REGARD  TO  PRODUCTIVITY 


Instructions:  Listed  in  Column  A  is  a  group  of  employees 
in  your  laboratory  whom  you  are  to  place  in  rank  order 
from  highest  to  lowest  with  regard  to  productivity  on  their 
specific  jobs.  These  men  do  different  kinds  of  work,  to  be 
sure,  but  you  are  to  rate  them  with  regard  to  productivity 
by  considering  their  individual  contributions  relative  to 
their  specific  assignments  during  the  period  that  you  have 
known  them. 

Think  of  their  total  contributions  whether  tangible  (publi¬ 
cations,  etc.),  or  intangible  (makes  useful  suggestions, 
stimulates  thinking  of  others,  etc.  ).  We  are  searching  for 
an  estimate  by  you  of  the  totai  contribution  that  the  people 
make  to  the  efforts  of  the  laboratory,  hence  their  overall 
productivity  BE  SURE  THAT  YOU  MAKE  YOUR  JUDG¬ 
MENTS  SOLELY  ON  THE  BASIS  OF  PRODUCTIVITY  AND 
CONTRIBUTIONS 

1  Cross  out  anyone  on  the  list  about  whom  you  have  no 
knowledge  and  therefore  cannot  rate.  Also  cross  our.  your 
own  name  if  it  appears  on  the  list 

2.  Look  at  Column  A  and  select  the  most  productive  per- 
son  there  Write  his  name  on  the  top  line  of  Column  B  . 
ITien  draw  a  line  through  his  name  in  Column  A 

3.  Select  the  most  productive  person  remaining  in  Column 
A,  write  his  name  in  Column  B  on  the  second  line,  and 
draw  a  line  through  his  name  in  Column  A 

4  Continue  this  process  until  all  of  the  names  have  been 
crossed  out  in  Column  A  and  placed  in  rank  order  with  re¬ 
gard  to  productivity  in  Column  B 

5  After  you  have  Finished  your  ranking  in  Column  B, 
place  a  check  mark  at  the  left  of  those  names  whom  you 
feel  you  have  been  able  to  rank  with  confidence . 

Rater  ______  _  _ _ _ 

Date 


-  For  Research  Purposes  Only  - 


Least 

Productive 


SOCIOMETRIC  RATING  SHEET 


Instructions :  You  are  to  consider  the, people  in  your  laboratory  when  you  respond  to  the  Items 
listed  below.  Read  each  item  carefully,  select  the  people  who  best  fit  the  demands  of  each  Item, 
and  write  their  names  on  the  appropriate  lines.  Choose  anyone  you  wish  from  your  laboratory  only. 

ITEM  1.  If  you  were  In  a  supervisory  position,  to  whom  would  you  readily  give  a  new  problem? 
Consider  the  ability  to  complete  the  work  effectively  and  ability  to  organize  work  well.  Although 
part  of  this  selection  will  ultimately  depend  upon  technical  skills  and  requirements,  try  now  to 
think  only  In  terms  of  the  ability  to  carry  out  research  and  complete  it  well.  (Omit  technicians. ) 

A.  List  no  more  than  three  people  to  whom  you  would  give  a  new  problem. 


B.  List  no  more  than  three  people  to  whom  you  would  most  likely  not  give  a  new  problem. 


ITEM  11.  With  whom  would  you  discuss  your  own  research  to  obtain  worthwhile  comments  and 
suggestions? 

A  List  no  more  than  three  people  with  whom  you  would  discuss  your  research  in  an  effort  to 
get  worthwhile  ideas. 


B.  List  no  more  than  three  people  with  whom  you  would  not  discuss  your  research  in  an  effort 
to  get  worthwhile  ideas. 


ITEM  111.  Consider  one's  ability  to  report  reliable  research  information. 

A.  List  no  more  than  three  people  whose  word  on  technical  matters  you  most  readily  accept. 


B.  List  no  more  than  three  people  whose  word  on  technical  matters  you  would  check  before 
accepting. 


Name  of  Rater  _ _ 

J 

( 


I 


Sample  Paqe;  Personal  Data  Form 


What  is  your  best  patent? 


10.  BEST  RESEARCH  REPORT:  Please  list  the  publication  or  other  research  report  that  you 
feel  is  the  best  one  you  have  yet  produced. 


11.  BEST  ACCOMPLISHMENTS:  Please  list  below  the  two  achievements  which  you  consider  your 
best  during  your  professional  career. 


12.  OUTSTANDING  SCIENTISTS  AT  E  RD:  Please  write  below  the  names  of  the  three  scientists 
in  E  R  D  who,  on  the  basis  of  your  own  judgment,  are  the  best.  Consider  their  productivity, 
success,  reputation,  etc. 


Please  write  below  the  names  of  the  three  best  scientists  in  your  laboratory.  Consider  their 
total  contributions  to  the  laboratory  efforts. 


This  is  part  of  an  important  experiment.  The  success  of  this  work 
depends  upon  the  full  cooperation  of  the  sample  selected  for  study. 
Please  return  this  personal  data  sheet  in  the  attached  envelope  (sealed) 
by  Tuesday,  March  18,  1958. 


Do  not  discuss  this  information 
with  anyone  except  the  Research 
Psychologist. 
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No  experimental  Small  expert-  Moderate  Large  ex-  Wry  substan- 

II.  What  »s  the  minimum  contribution  to  t|ie  contributions.  mental  contri-  experimental  perimental  tial  expert- 

expertmintai  development  of  your  specialty  All  theoretical  butlon  contribution.  contribution.  mental  contri- 

in  science  with  which  you  would  be  satisfied  or  administrative.  bution. 

during  your  professional  career?. ...  .  (  )  (  )  (  )  (  )  (  ) 


Sample  Page:  Biographical  Information  Elank 


40.  Which  of  the  following  best  describes 
how  you  felt  about  your  social  skills 
during  your  adolescence? 

A.  Definitely  below  average 
B  Slightly  below  average. 

C.  About  average. 

D.  Slightly  above  average. 

E.  Definitely  above  average. 

41  .  During  your  childhood  and  adolescence, 
how  old  were  the  people  you  generally 
associated  with? 

A  Older  than  myself. 

B  .  My  own  age  or  older. 

C.  My  own  age. 

D.  My  own  age  or  younger. 

E.  Younger  than  myself 

42.  As  a  youth  how  often  did  you  confer  with 
your  parents  or  other  adults  about  your 
occupational  choice? 

A.  Frequently. 

B  .  Occasionally. 

C.  Seldom 

D.  Never. 

43.  Between  the  ages  of  12  and  18,  who  had 
the  most  influence  on  your  occupational 
choice? 

A.  Father. 

B  .  Mother. 

C  An  idol 

D.  A  friend,  ora  relative. 

E.  None  of  the  above. 

44.  In  senior  high  school  how  definite  were 
you  about  your  future  occupation? 

A.  Definite  enough  to  make  every 
effort  toward  the  specific  goal. 

B  .  Had  a  good  idea,  but  worked 
toward  a  general  goal. 

C.  Had  some  ideas,  but  preferred  to 
remain  somewhat  undecided. 

D.  Had  no  ideas  whatsoever. 

45.  Altogether,  how  long  did  you  live  away 
from  home  up  to  age  17? 

A  1  month  or  less. 

B  .  1  to  6  months  . 

C.  6  months  to  1  year. 

D.  1  to  4  years. 

E.  More  than  4  years. 


46.  In  general,  how  much  education  did 
your  high  school  friends  think  was  nec¬ 
essary  for  future  success  and  security? 

A.  Some  high  school. 

B  .  A  high  school  degree. 

C  A  college  degree. 

D  A  graduate  degree. 

E.  Don't  know. 

47.  Where  did  you  live  during  most  of 
your  undergraduate  years  at  college? 

A.  Home. 

B  College  dorm. 

C.  Boarding  house. 

D  Fraternity  house. 

E.  None  of  the  above . 

48.  On  the  average  while  in  college,  how 
many  hours  a  week  did  you  spend  in 
reading?  (Excluding  class  work.) 

A.  None. 

B.  1  to  5. 

C.  6  to  10 

D.  10  to  15. 

E .  Over  1 5  hours . 

49.  About  how  often  did  you  have  dates  during 
your  last  two  years  (Jr.  and  Sr.  years) 

in  college? 

A.  Once  or  twice  a  year  or  less. 

B  .  From  3  to  11  times  a  year. 

C.  From  1  to  3  times  a  month. 

D.  Once  or  twice  a  week. 

E.  Three  or  more  times  a  week. 

50.  During  your  undergraduate  college  work, 
if  several  conflicting  activities  arose, 
which cf  the  followinggenerally  won  out? 

A.  My  social  life--dates,  movies,  etc. 

B  .  My  studies 

C.  Work  outside  of  school. 

D  Athletics. 

E.  Other  outside  activities 

51.  When  you  were  about  20  years  old,  how 
important  was  it  to  you  to  be  well 
accepted  in  a  social  group? 

A.  Very  important. 

B  .  Quite  important . 

C.  Indifferent 

D.  Rather  unimportant. 

E.  Very  unimportant 
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